2019-12-22T21:58:15.923 INFO:root:teuthology version: 1.0.0-744c324 2019-12-22T21:58:15.924 DEBUG:teuthology.run:Teuthology command: teuthology /tmp/teuthology-worker.gFCnd0.tmp -- --verbose --lock --description rados:thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/luminous.yaml backoff/peering.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/crush-compat.yaml distro$/{centos_7.6.yaml} msgr-failures/osd-delay.yaml rados.yaml thrashers/pggrow.yaml thrashosds-health.yaml u.yaml workloads/radosbench.yaml} --name sage-2019-12-22_21:54:24-rados:thrash-old-clients-wip-sage-testing-2019-12-22-0931-distro-basic-smithi --block --owner scheduled_sage@teuthology --archive /home/teuthworker/archive/sage-2019-12-22_21:54:24-rados:thrash-old-clients-wip-sage-testing-2019-12-22-0931-distro-basic-smithi/4626400 2019-12-22T21:58:16.036 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-12-22T21:58:16.151 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/sage-2019-12-22_21:54:24-rados:thrash-old-clients-wip-sage-testing-2019-12-22-0931-distro-basic-smithi/4626400 branch: wip-sage-testing-2019-12-22-0931 description: rados:thrash-old-clients/{0-size-min-size-overrides/2-size-2-min-size.yaml 1-install/luminous.yaml backoff/peering.yaml ceph.yaml clusters/{openstack.yaml three-plus-one.yaml} d-balancer/crush-compat.yaml distro$/{centos_7.6.yaml} msgr-failures/osd-delay.yaml rados.yaml thrashers/pggrow.yaml thrashosds-health.yaml u.yaml workloads/radosbench.yaml} email: null first_in_suite: false job_id: '4626400' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: sage-2019-12-22_21:54:24-rados:thrash-old-clients-wip-sage-testing-2019-12-22-0931-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 30 - volumes: count: 4 size: 10 os_type: centos os_version: '7.6' overrides: admin_socket: branch: wip-sage-testing-2019-12-22-0931 ceph: conf: client.2: debug ms: 1 debug objecter: 20 debug rados: 20 global: mon client directed command retry: 5 mon warn on msgr2 not enabled: false ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: osd ms inject internal delays: 0.002 ms inject socket failures: 2500 osd_pool_default_min_size: 2 osd_pool_default_size: 2 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug filestore: 20 debug journal: 20 debug ms: 1 debug osd: 25 filestore odsync write: true osd backoff on peering: true osd debug verify cached snaps: true osd debug verify missing on start: true osd max backfills: 2 osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 0.5 log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(REQUEST_STUCK\) - \(MON_DOWN\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow requests mon_bind_msgr2: false sha1: 99b509ffd4dd354919b08b38ba83b998fbfdc107 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: 99b509ffd4dd354919b08b38ba83b998fbfdc107 workunit: branch: wip-old-clients-cephadm sha1: abe4c1c94873149a777c88f919a579d57de0f422 owner: scheduled_sage@teuthology priority: 30 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - - mon.c - osd.8 - osd.9 - osd.10 - osd.11 - client.2 sha1: 99b509ffd4dd354919b08b38ba83b998fbfdc107 suite: rados:thrash-old-clients suite_branch: wip-old-clients-cephadm suite_path: /home/teuthworker/src/github.com_liewegas_ceph_wip-old-clients-cephadm/qa suite_relpath: qa suite_repo: https://github.com/liewegas/ceph suite_sha1: abe4c1c94873149a777c88f919a579d57de0f422 tasks: - install: branch: luminous exclude_packages: - librados3 - ceph-mgr-dashboard - ceph-mgr-diskprediction-local - ceph-mgr-diskprediction-cloud - ceph-mgr-rook - ceph-mgr-cephadm - cephadm - ceph-immutable-object-cache - ceph-base - python3-rados - python3-rgw - python3-rbd - python3-cephfs - librados-devel extra_packages: - librados2 - python-rados - python-rgw - python-rbd - python-cephfs - cephadm: null - exec: mon.a: - while ! ceph balancer status ; do sleep 1 ; done - ceph balancer mode crush-compat - ceph balancer on - thrashosds: chance_pgnum_grow: 2 chance_pgpnum_fix: 1 timeout: 1200 - sleep: duration: 600000 - full_sequential: - radosbench: clients: - client.2 time: 90 - radosbench: clients: - client.2 time: 90 - radosbench: clients: - client.2 time: 90 - radosbench: clients: - client.2 time: 90 - radosbench: clients: - client.2 time: 90 - radosbench: clients: - client.2 time: 90 - radosbench: clients: - client.2 time: 90 - radosbench: clients: - client.2 time: 90 teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.22731 2019-12-22T21:58:16.151 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_liewegas_ceph_wip-old-clients-cephadm/qa; will attempt to use it 2019-12-22T21:58:16.153 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_liewegas_ceph_wip-old-clients-cephadm/qa/tasks 2019-12-22T21:58:16.154 INFO:teuthology.run_tasks:Running task internal.check_packages... 2019-12-22T21:58:16.155 INFO:teuthology.task.internal:Checking packages... 2019-12-22T21:58:16.188 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'basic' and ceph hash '99b509ffd4dd354919b08b38ba83b998fbfdc107' 2019-12-22T21:58:16.188 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2019-12-22T21:58:16.189 INFO:teuthology.packaging:ref: None 2019-12-22T21:58:16.189 INFO:teuthology.packaging:tag: None 2019-12-22T21:58:16.189 INFO:teuthology.packaging:branch: wip-sage-testing-2019-12-22-0931 2019-12-22T21:58:16.189 INFO:teuthology.packaging:sha1: 99b509ffd4dd354919b08b38ba83b998fbfdc107 2019-12-22T21:58:16.189 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F7%2Fx86_64&ref=wip-sage-testing-2019-12-22-0931 2019-12-22T21:58:16.358 INFO:teuthology.task.internal:Found packages for ceph version 15.0.0-8807.g99b509f 2019-12-22T21:58:16.358 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2019-12-22T21:58:16.376 INFO:teuthology.task.internal:no buildpackages task found 2019-12-22T21:58:16.376 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2019-12-22T21:58:16.428 INFO:teuthology.task.internal.lock_machines:Locking machines... 2019-12-22T21:58:16.428 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-12-22T21:58:16.479 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2019-12-22T21:58:26.503 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2019-12-22T21:58:36.529 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2019-12-22T21:58:46.547 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2019-12-22T21:58:56.565 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2019-12-22T21:59:06.593 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2019-12-22T21:59:16.628 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2019-12-22T21:59:26.655 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2019-12-22T21:59:36.673 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2019-12-22T21:59:46.690 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2019-12-22T21:59:56.703 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2019-12-22T22:00:06.722 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2019-12-22T22:00:16.737 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2019-12-22T22:00:26.760 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 3)... 2019-12-22T22:00:36.777 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 4)... 2019-12-22T22:00:46.792 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2019-12-22T22:00:56.813 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2019-12-22T22:01:06.828 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 3, have 2)... 2019-12-22T22:01:16.847 DEBUG:teuthology.lock.ops:lock_many request: {'count': 3, 'locked_by': 'scheduled_sage@teuthology', 'description': '/home/teuthworker/archive/sage-2019-12-22_21:54:24-rados:thrash-old-clients-wip-sage-testing-2019-12-22-0931-distro-basic-smithi/4626400', 'machine_type': 'smithi'} 2019-12-22T22:01:16.866 DEBUG:teuthology.lock.ops:locked ubuntu@smithi091.front.sepia.ceph.com, ubuntu@smithi107.front.sepia.ceph.com, ubuntu@smithi043.front.sepia.ceph.com 2019-12-22T22:01:17.015 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2019-12-22T22:01:17.272 INFO:teuthology.provision.fog.smithi107:Scheduling deploy of centos 7.6 2019-12-22T22:01:17.283 INFO:teuthology.provision.fog.smithi091:Scheduling deploy of centos 7.6 2019-12-22T22:01:17.286 INFO:teuthology.provision.fog.smithi043:Scheduling deploy of centos 7.6 2019-12-22T22:01:17.659 INFO:teuthology.orchestra.console:Power off smithi091 2019-12-22T22:01:17.660 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-12-22T22:01:17.689 INFO:teuthology.orchestra.console:Power off smithi107 2019-12-22T22:01:17.689 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-12-22T22:01:17.700 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-22T22:01:17.719 INFO:teuthology.orchestra.console:Power off smithi043 2019-12-22T22:01:17.720 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-12-22T22:01:17.739 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-22T22:01:17.758 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-22T22:01:17.841 INFO:teuthology.orchestra.console:Power off for smithi091 completed 2019-12-22T22:01:17.870 INFO:teuthology.orchestra.console:Power off for smithi107 completed 2019-12-22T22:01:17.881 INFO:teuthology.orchestra.console:Power off for smithi043 completed 2019-12-22T22:01:17.942 INFO:teuthology.orchestra.console:Power on smithi091 2019-12-22T22:01:17.942 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-12-22T22:01:17.970 INFO:teuthology.orchestra.console:Power on smithi107 2019-12-22T22:01:17.971 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-12-22T22:01:17.982 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-22T22:01:17.996 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-22T22:01:18.005 INFO:teuthology.orchestra.console:Power on smithi043 2019-12-22T22:01:18.006 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-12-22T22:01:18.030 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-22T22:01:22.022 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-22T22:01:22.030 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-22T22:01:22.054 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-22T22:01:26.137 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi091.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-22T22:01:26.151 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi107.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-22T22:01:26.167 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-22T22:01:26.351 INFO:teuthology.orchestra.console:Power on for smithi091 completed 2019-12-22T22:01:26.361 INFO:teuthology.orchestra.console:Power on for smithi107 completed 2019-12-22T22:01:26.416 INFO:teuthology.orchestra.console:Power on for smithi043 completed 2019-12-22T22:01:26.454 INFO:teuthology.provision.fog.smithi091:Waiting for deploy to finish 2019-12-22T22:01:26.462 INFO:teuthology.provision.fog.smithi107:Waiting for deploy to finish 2019-12-22T22:01:26.517 INFO:teuthology.provision.fog.smithi043:Waiting for deploy to finish 2019-12-22T22:03:58.076 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi043.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:04:07.181 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi043.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:04:13.133 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi091.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:04:16.234 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi043.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:04:22.218 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi091.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:04:25.289 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi043.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:04:28.258 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi107.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:04:31.274 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi091.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:04:34.346 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi043.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:04:37.354 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi107.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:04:40.332 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi091.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:04:43.405 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi043.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:04:46.411 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi107.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:04:49.389 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi091.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:04:55.468 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi107.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:04:58.445 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi091.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:05:04.524 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi107.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:05:07.498 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi091.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:05:13.578 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi107.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:05:16.554 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi091.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:05:22.637 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi107.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:05:31.694 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi107.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:05:40.757 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi107.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:05:49.409 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi043.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:05:49.637 INFO:teuthology.orchestra.run.smithi043:> hostname 2019-12-22T22:05:50.210 INFO:teuthology.orchestra.run.smithi043.stdout:smithi043.front.sepia.ceph.com 2019-12-22T22:05:50.210 INFO:teuthology.orchestra.run.smithi043:> grep smithi043.front.sepia.ceph.com /etc/hosts 2019-12-22T22:05:50.304 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43 smithi043.front.sepia.ceph.com smithi043 2019-12-22T22:05:50.305 INFO:teuthology.orchestra.run.smithi043:> sudo hostname smithi043 2019-12-22T22:05:50.477 INFO:teuthology.orchestra.run.smithi043:> sudo sed -i -e 's/smithi043.front.sepia.ceph.com/smithi043/g' /etc/hosts 2019-12-22T22:05:50.584 INFO:teuthology.orchestra.run.smithi043:> sudo sed -i -e 's/172.21.15.43/172.21.15.43/g' /etc/hosts 2019-12-22T22:05:50.680 INFO:teuthology.orchestra.run.smithi043:> sudo sed -i -e 's/smithi043.front.sepia.ceph.com/smithi043/g' /etc/hostname 2019-12-22T22:05:50.784 INFO:teuthology.orchestra.run.smithi043:> sudo hostname smithi043 2019-12-22T22:05:50.889 INFO:teuthology.provision.fog.smithi043:Deploy complete! 2019-12-22T22:05:50.891 DEBUG:teuthology.parallel:result is None 2019-12-22T22:06:22.582 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi091.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:06:22.781 INFO:teuthology.orchestra.run.smithi091:> hostname 2019-12-22T22:06:23.333 INFO:teuthology.orchestra.run.smithi091.stdout:smithi091.front.sepia.ceph.com 2019-12-22T22:06:23.334 INFO:teuthology.orchestra.run.smithi091:> grep smithi091.front.sepia.ceph.com /etc/hosts 2019-12-22T22:06:23.421 INFO:teuthology.orchestra.run.smithi091.stdout:172.21.15.91 smithi091.front.sepia.ceph.com smithi091 2019-12-22T22:06:23.421 INFO:teuthology.orchestra.run.smithi091:> sudo hostname smithi091 2019-12-22T22:06:23.562 INFO:teuthology.orchestra.run.smithi091:> sudo sed -i -e 's/smithi091.front.sepia.ceph.com/smithi091/g' /etc/hosts 2019-12-22T22:06:23.672 INFO:teuthology.orchestra.run.smithi091:> sudo sed -i -e 's/172.21.15.91/172.21.15.91/g' /etc/hosts 2019-12-22T22:06:23.758 INFO:teuthology.orchestra.run.smithi091:> sudo sed -i -e 's/smithi091.front.sepia.ceph.com/smithi091/g' /etc/hostname 2019-12-22T22:06:23.830 INFO:teuthology.orchestra.run.smithi091:> sudo hostname smithi091 2019-12-22T22:06:23.893 INFO:teuthology.provision.fog.smithi091:Deploy complete! 2019-12-22T22:06:23.894 DEBUG:teuthology.parallel:result is None 2019-12-22T22:06:46.774 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi107.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:06:47.004 INFO:teuthology.orchestra.run.smithi107:> hostname 2019-12-22T22:06:47.589 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107 2019-12-22T22:06:47.590 INFO:teuthology.orchestra.run.smithi107:> grep smithi107 /etc/hosts 2019-12-22T22:06:47.679 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107 smithi107.front.sepia.ceph.com smithi107 2019-12-22T22:06:47.679 INFO:teuthology.orchestra.run.smithi107:> sudo hostname smithi107 2019-12-22T22:06:47.825 INFO:teuthology.orchestra.run.smithi107:> sudo sed -i -e 's/smithi107/smithi107/g' /etc/hosts 2019-12-22T22:06:47.941 INFO:teuthology.orchestra.run.smithi107:> sudo sed -i -e 's/172.21.15.107/172.21.15.107/g' /etc/hosts 2019-12-22T22:06:48.032 INFO:teuthology.orchestra.run.smithi107:> sudo sed -i -e 's/smithi107/smithi107/g' /etc/hostname 2019-12-22T22:06:48.122 INFO:teuthology.orchestra.run.smithi107:> sudo hostname smithi107 2019-12-22T22:06:48.205 INFO:teuthology.provision.fog.smithi107:Deploy complete! 2019-12-22T22:06:48.207 DEBUG:teuthology.parallel:result is None 2019-12-22T22:06:48.439 INFO:teuthology.lock.ops:Checking smithi107.front.sepia.ceph.com 2019-12-22T22:06:48.439 INFO:teuthology.lock.ops:New key found. Updating... 2019-12-22T22:06:48.453 INFO:teuthology.lock.ops:Checking smithi091.front.sepia.ceph.com 2019-12-22T22:06:48.454 INFO:teuthology.lock.ops:New key found. Updating... 2019-12-22T22:06:48.473 INFO:teuthology.lock.ops:Checking smithi043.front.sepia.ceph.com 2019-12-22T22:06:48.474 INFO:teuthology.lock.ops:New key found. Updating... 2019-12-22T22:06:48.494 INFO:teuthology.lock.ops:Updating [smithi107.front.sepia.ceph.com]: set os type and version on server 2019-12-22T22:06:48.494 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-22T22:06:48.495 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi107.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:06:48.590 INFO:teuthology.orchestra.run.smithi107:> true 2019-12-22T22:06:48.984 INFO:teuthology.orchestra.run.smithi107:> uname -m 2019-12-22T22:06:49.054 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2019-12-22T22:06:49.054 INFO:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2019-12-22T22:06:49.130 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Linux" 2019-12-22T22:06:49.130 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="7 (Core)" 2019-12-22T22:06:49.130 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2019-12-22T22:06:49.130 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2019-12-22T22:06:49.130 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="7" 2019-12-22T22:06:49.131 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-22T22:06:49.131 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2019-12-22T22:06:49.131 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-22T22:06:49.131 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://www.centos.org/" 2019-12-22T22:06:49.131 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-22T22:06:49.131 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:06:49.131 INFO:teuthology.orchestra.run.smithi107.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-22T22:06:49.132 INFO:teuthology.orchestra.run.smithi107.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-22T22:06:49.132 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-22T22:06:49.132 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-22T22:06:49.132 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:06:49.133 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2019-12-22T22:06:49.146 INFO:teuthology.lock.ops:Updating [smithi091.front.sepia.ceph.com]: set os type and version on server 2019-12-22T22:06:49.155 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-22T22:06:49.156 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi091.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:06:49.252 INFO:teuthology.orchestra.run.smithi091:> true 2019-12-22T22:06:49.632 INFO:teuthology.orchestra.run.smithi091:> uname -m 2019-12-22T22:06:49.694 INFO:teuthology.orchestra.run.smithi091.stdout:x86_64 2019-12-22T22:06:49.695 INFO:teuthology.orchestra.run.smithi091:> cat /etc/os-release 2019-12-22T22:06:49.768 INFO:teuthology.orchestra.run.smithi091.stdout:NAME="CentOS Linux" 2019-12-22T22:06:49.769 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION="7 (Core)" 2019-12-22T22:06:49.769 INFO:teuthology.orchestra.run.smithi091.stdout:ID="centos" 2019-12-22T22:06:49.769 INFO:teuthology.orchestra.run.smithi091.stdout:ID_LIKE="rhel fedora" 2019-12-22T22:06:49.769 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION_ID="7" 2019-12-22T22:06:49.769 INFO:teuthology.orchestra.run.smithi091.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-22T22:06:49.769 INFO:teuthology.orchestra.run.smithi091.stdout:ANSI_COLOR="0;31" 2019-12-22T22:06:49.769 INFO:teuthology.orchestra.run.smithi091.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-22T22:06:49.770 INFO:teuthology.orchestra.run.smithi091.stdout:HOME_URL="https://www.centos.org/" 2019-12-22T22:06:49.770 INFO:teuthology.orchestra.run.smithi091.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-22T22:06:49.770 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:06:49.770 INFO:teuthology.orchestra.run.smithi091.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-22T22:06:49.770 INFO:teuthology.orchestra.run.smithi091.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-22T22:06:49.770 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-22T22:06:49.770 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-22T22:06:49.770 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:06:49.771 INFO:teuthology.lock.ops:Updating smithi091.front.sepia.ceph.com on lock server 2019-12-22T22:06:49.784 INFO:teuthology.lock.ops:Updating [smithi043.front.sepia.ceph.com]: set os type and version on server 2019-12-22T22:06:49.784 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-22T22:06:49.784 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi043.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:06:49.883 INFO:teuthology.orchestra.run.smithi043:> true 2019-12-22T22:06:50.276 INFO:teuthology.orchestra.run.smithi043:> uname -m 2019-12-22T22:06:50.346 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2019-12-22T22:06:50.346 INFO:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2019-12-22T22:06:50.421 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Linux" 2019-12-22T22:06:50.422 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="7 (Core)" 2019-12-22T22:06:50.422 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2019-12-22T22:06:50.422 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2019-12-22T22:06:50.422 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="7" 2019-12-22T22:06:50.422 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-22T22:06:50.422 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2019-12-22T22:06:50.423 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-22T22:06:50.423 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://www.centos.org/" 2019-12-22T22:06:50.423 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-22T22:06:50.423 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:06:50.423 INFO:teuthology.orchestra.run.smithi043.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-22T22:06:50.423 INFO:teuthology.orchestra.run.smithi043.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-22T22:06:50.424 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-22T22:06:50.424 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-22T22:06:50.424 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:06:50.424 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2019-12-22T22:06:50.450 INFO:teuthology.task.internal.lock_machines:3 smithi machines locked this try, 3/3 locked so far 2019-12-22T22:06:50.506 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi043.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC3H4CgHxXVxWF6vDzCFdGaST0VogqFT6E7LI5j9lDEkCmrMOXfMH0pv4GOPJ+ZCPVgZA4jD+6m5FVecAUFbBCBI3HZPUPP5kFjw1nFeggdCECzN1hbsepaLbvi1U6PumzNuRFJJJaCi2eN9k35WBh0ACCne5Tqo+U6BxCZFSr+F39BNjLpZ7e13uc9ExasAUN2xXcStZgrNJs6tmv74CoqgqDmYCDw2GeuySFeXpMkGBW91QnmbER5TBBO7HNqv3upQw3/26dn7v5AQpmU/ZUry6vIBgixttvEdj35AMslC4wRxTCgueLOlGlY83Yfv5kXRrp2DaZEr103YVlC1xDx smithi091.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNriNQD6dZc6GTQVB4Yoxgf+n/ZkLBJ6iy6kRmjFl82fYp8/sJ03PJCgnimXYdhCwWcbeXtmFTlaExU4D46Z5WVqQVegGgOJX20Nx4PBCeTGx29vigIPjpPKVMI4yN3jGNtENOo/MMjP/bM8bLaeHwWc5uHyCUT2oVVESwNEVjWHeH99peLyH7gm5fmzzFL9b1qQWUGWLN1UPgeZUkgiEjodzs2ZtKmXz1ywTM1BtYyBxgz/dll0OGKwlw3fl3IyQSD4WIphE6QvLWVBeLemq/mlWKd+aqzzq/yXarlZGxLqKxlQWSCRQYJ4o/u+RndiKS+zb6JO7jKxB8elH3JUlr smithi107.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDDIStnHvotAPrmBWk3WGjUhsbJs/yvQ/1WfhwZsmseUe6t/6/7yNKTqE9BXEaJOF7CLkHI5zKLjIAdUuCv7dkHmxi2NISzKM0AVl0vek24TY6ZLge2g/tl3i1KVr9bhdwDrV4cw0JiyjPUAEWFh2dK0xUbHz1zNu1KIMrvZyS+PuAq0XBKL7i0erM3xVJTUZZj0kOCkDBxyj20oe+/ShCjeQXp2kXK46FFqOvI76ptzxqw9bvuIJHU8OxSOwMjDUSW8G40wvlybz8fN+mT0P+clh0Xl8USYnShdibpjDa7ia0eaux8cwhOQi/7NwXCUGFhcn73uXOYMAmO65EnrsP 2019-12-22T22:06:50.506 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-12-22T22:06:50.558 INFO:teuthology.run_tasks:Running task internal.save_config... 2019-12-22T22:06:50.575 INFO:teuthology.task.internal:Saving configuration 2019-12-22T22:06:50.598 INFO:teuthology.run_tasks:Running task internal.check_lock... 2019-12-22T22:06:50.621 INFO:teuthology.task.internal.check_lock:Checking locks... 2019-12-22T22:06:50.655 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2019-12-22 22:01:16.855072', u'locked_by': u'scheduled_sage@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi091.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJvqPib3ZQXtCHq8rXs3tyszaTwEAKnmM4h8iH63JLv0tY9t7gxRPAfu18RsuF9rwrj+3f9qSQGO6ujAxw7nzT4=', u'description': u'/home/teuthworker/archive/sage-2019-12-22_21:54:24-rados:thrash-old-clients-wip-sage-testing-2019-12-22-0931-distro-basic-smithi/4626400'} 2019-12-22T22:06:50.696 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2019-12-22 22:01:16.855092', u'locked_by': u'scheduled_sage@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi107.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLE4bFdwKlrzM6okY6oQkb1X4PJCXzaAMyT3ddnjP4FXNvnw7QbYG3BhsLmg4NjmhSbDWPgd8mTaKVoR3fyplpc=', u'description': u'/home/teuthworker/archive/sage-2019-12-22_21:54:24-rados:thrash-old-clients-wip-sage-testing-2019-12-22-0931-distro-basic-smithi/4626400'} 2019-12-22T22:06:50.716 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2019-12-22 22:01:16.855107', u'locked_by': u'scheduled_sage@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi043.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHBEW08ALwIXNUanQl+EEpQcTUYxFV46s2foKws3YEZGdYQY6ue/ZmZ6jHbiZQpUVux6SxWW/YYbTQdqozVKU1w=', u'description': u'/home/teuthworker/archive/sage-2019-12-22_21:54:24-rados:thrash-old-clients-wip-sage-testing-2019-12-22-0931-distro-basic-smithi/4626400'} 2019-12-22T22:06:50.716 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2019-12-22T22:06:50.740 INFO:teuthology.task.internal:roles: ubuntu@smithi091.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2019-12-22T22:06:50.740 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2019-12-22T22:06:50.740 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.c', 'osd.8', 'osd.9', 'osd.10', 'osd.11', 'client.2'] 2019-12-22T22:06:50.741 INFO:teuthology.run_tasks:Running task console_log... 2019-12-22T22:06:50.894 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2019-12-22T22:06:50.895 INFO:teuthology.run_tasks:Running task internal.connect... 2019-12-22T22:06:50.944 INFO:teuthology.task.internal:Opening connections... 2019-12-22T22:06:50.944 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi091.front.sepia.ceph.com 2019-12-22T22:06:50.946 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi091.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:06:51.049 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2019-12-22T22:06:51.050 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi043.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:06:51.156 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2019-12-22T22:06:51.157 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi107.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:06:51.265 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2019-12-22T22:06:51.296 INFO:teuthology.orchestra.run.smithi091:> uname -m 2019-12-22T22:06:51.382 INFO:teuthology.orchestra.run.smithi091.stdout:x86_64 2019-12-22T22:06:51.382 INFO:teuthology.orchestra.run.smithi091:> cat /etc/os-release 2019-12-22T22:06:51.437 INFO:teuthology.orchestra.run.smithi091.stdout:NAME="CentOS Linux" 2019-12-22T22:06:51.437 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION="7 (Core)" 2019-12-22T22:06:51.437 INFO:teuthology.orchestra.run.smithi091.stdout:ID="centos" 2019-12-22T22:06:51.437 INFO:teuthology.orchestra.run.smithi091.stdout:ID_LIKE="rhel fedora" 2019-12-22T22:06:51.438 INFO:teuthology.orchestra.run.smithi091.stdout:VERSION_ID="7" 2019-12-22T22:06:51.438 INFO:teuthology.orchestra.run.smithi091.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-22T22:06:51.439 INFO:teuthology.orchestra.run.smithi091.stdout:ANSI_COLOR="0;31" 2019-12-22T22:06:51.439 INFO:teuthology.orchestra.run.smithi091.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-22T22:06:51.439 INFO:teuthology.orchestra.run.smithi091.stdout:HOME_URL="https://www.centos.org/" 2019-12-22T22:06:51.439 INFO:teuthology.orchestra.run.smithi091.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-22T22:06:51.440 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:06:51.440 INFO:teuthology.orchestra.run.smithi091.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-22T22:06:51.440 INFO:teuthology.orchestra.run.smithi091.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-22T22:06:51.441 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-22T22:06:51.441 INFO:teuthology.orchestra.run.smithi091.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-22T22:06:51.441 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:06:51.442 INFO:teuthology.lock.ops:Updating smithi091.front.sepia.ceph.com on lock server 2019-12-22T22:06:51.463 INFO:teuthology.orchestra.run.smithi043:> uname -m 2019-12-22T22:06:51.586 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2019-12-22T22:06:51.599 INFO:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2019-12-22T22:06:51.685 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Linux" 2019-12-22T22:06:51.685 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="7 (Core)" 2019-12-22T22:06:51.685 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2019-12-22T22:06:51.685 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2019-12-22T22:06:51.685 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="7" 2019-12-22T22:06:51.685 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-22T22:06:51.686 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2019-12-22T22:06:51.686 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-22T22:06:51.686 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://www.centos.org/" 2019-12-22T22:06:51.686 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-22T22:06:51.686 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:06:51.686 INFO:teuthology.orchestra.run.smithi043.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-22T22:06:51.686 INFO:teuthology.orchestra.run.smithi043.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-22T22:06:51.686 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-22T22:06:51.687 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-22T22:06:51.687 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:06:51.687 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2019-12-22T22:06:51.700 INFO:teuthology.orchestra.run.smithi107:> uname -m 2019-12-22T22:06:51.817 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2019-12-22T22:06:51.818 INFO:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2019-12-22T22:06:51.894 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Linux" 2019-12-22T22:06:51.894 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="7 (Core)" 2019-12-22T22:06:51.894 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2019-12-22T22:06:51.894 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2019-12-22T22:06:51.894 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="7" 2019-12-22T22:06:51.895 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-22T22:06:51.895 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2019-12-22T22:06:51.895 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-22T22:06:51.895 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://www.centos.org/" 2019-12-22T22:06:51.895 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-22T22:06:51.896 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:06:51.896 INFO:teuthology.orchestra.run.smithi107.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-22T22:06:51.896 INFO:teuthology.orchestra.run.smithi107.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-22T22:06:51.896 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-22T22:06:51.896 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-22T22:06:51.897 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:06:51.897 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2019-12-22T22:06:51.910 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2019-12-22T22:06:51.982 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2019-12-22T22:06:52.015 INFO:teuthology.task.internal:Checking for old test directory... 2019-12-22T22:06:52.015 INFO:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2019-12-22T22:06:52.041 INFO:teuthology.orchestra.run.smithi091:> test '!' -e /home/ubuntu/cephtest 2019-12-22T22:06:52.069 INFO:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2019-12-22T22:06:52.174 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2019-12-22T22:06:52.387 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2019-12-22T22:06:52.388 INFO:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2019-12-22T22:06:52.392 INFO:teuthology.orchestra.run.smithi091:> test -z $(ls -A /var/lib/ceph) 2019-12-22T22:06:52.396 INFO:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2019-12-22T22:06:52.510 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2019-12-22T22:06:52.665 INFO:teuthology.run_tasks:Running task kernel... 2019-12-22T22:06:52.799 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'osd.8': {'sha1': 'distro', 'kdb': True}, 'mon.c': {'sha1': 'distro', 'kdb': True}, 'client.0': {'sha1': 'distro', 'kdb': True}, 'osd.1': {'sha1': 'distro', 'kdb': True}, 'osd.0': {'sha1': 'distro', 'kdb': True}, 'osd.3': {'sha1': 'distro', 'kdb': True}, 'osd.2': {'sha1': 'distro', 'kdb': True}, 'osd.5': {'sha1': 'distro', 'kdb': True}, 'osd.9': {'sha1': 'distro', 'kdb': True}, 'osd.7': {'sha1': 'distro', 'kdb': True}, 'osd.4': {'sha1': 'distro', 'kdb': True}, 'osd.11': {'sha1': 'distro', 'kdb': True}, 'osd.10': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}, 'mgr.y': {'sha1': 'distro', 'kdb': True}, 'mgr.x': {'sha1': 'distro', 'kdb': True}, 'client.2': {'sha1': 'distro', 'kdb': True}, 'osd.6': {'sha1': 'distro', 'kdb': True}, 'client.1': {'sha1': 'distro', 'kdb': True}} 2019-12-22T22:06:52.799 INFO:teuthology.task.kernel:config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'mon.c': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}}, timeout 300 2019-12-22T22:06:52.800 INFO:teuthology.orchestra.run.smithi091:> uname -r 2019-12-22T22:06:52.888 INFO:teuthology.orchestra.run.smithi091.stdout:3.10.0-1062.4.1.el7.x86_64 2019-12-22T22:06:52.889 INFO:teuthology.task.kernel:Running kernel on smithi091: 3.10.0-1062.4.1.el7.x86_64 2019-12-22T22:06:52.889 INFO:teuthology.orchestra.run.smithi091:> sudo yum install -y kernel 2019-12-22T22:06:53.699 INFO:teuthology.orchestra.run.smithi091.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:06:53.811 INFO:teuthology.orchestra.run.smithi091.stdout:Determining fastest mirrors 2019-12-22T22:06:54.162 INFO:teuthology.orchestra.run.smithi091.stdout: * base: mirror.umd.edu 2019-12-22T22:06:54.174 INFO:teuthology.orchestra.run.smithi091.stdout: * epel: ftp.linux.ncsu.edu 2019-12-22T22:06:54.175 INFO:teuthology.orchestra.run.smithi091.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:06:54.175 INFO:teuthology.orchestra.run.smithi091.stdout: * updates: mirror.wdc1.us.leaseweb.net 2019-12-22T22:06:57.342 INFO:teuthology.orchestra.run.smithi091.stdout:Resolving Dependencies 2019-12-22T22:06:57.342 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:06:57.343 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package kernel.x86_64 0:3.10.0-1062.9.1.el7 will be installed 2019-12-22T22:06:58.654 INFO:teuthology.orchestra.run.smithi091.stdout:--> Finished Dependency Resolution 2019-12-22T22:06:59.517 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:06:59.517 INFO:teuthology.orchestra.run.smithi091.stdout:Dependencies Resolved 2019-12-22T22:06:59.531 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:06:59.531 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:06:59.531 INFO:teuthology.orchestra.run.smithi091.stdout: Package Arch Version Repository Size 2019-12-22T22:06:59.532 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:06:59.532 INFO:teuthology.orchestra.run.smithi091.stdout:Installing: 2019-12-22T22:06:59.532 INFO:teuthology.orchestra.run.smithi091.stdout: kernel x86_64 3.10.0-1062.9.1.el7 updates 50 M 2019-12-22T22:06:59.532 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:06:59.532 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction Summary 2019-12-22T22:06:59.532 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:06:59.532 INFO:teuthology.orchestra.run.smithi091.stdout:Install 1 Package 2019-12-22T22:06:59.533 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:06:59.533 INFO:teuthology.orchestra.run.smithi091.stdout:Total download size: 50 M 2019-12-22T22:06:59.533 INFO:teuthology.orchestra.run.smithi091.stdout:Installed size: 64 M 2019-12-22T22:06:59.533 INFO:teuthology.orchestra.run.smithi091.stdout:Downloading packages: 2019-12-22T22:06:59.541 INFO:teuthology.orchestra.run.smithi091.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-22T22:07:00.480 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction check 2019-12-22T22:07:00.655 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction test 2019-12-22T22:07:01.638 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction test succeeded 2019-12-22T22:07:01.638 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction 2019-12-22T22:07:36.726 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-22T22:07:37.271 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-22T22:07:37.272 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:07:37.272 INFO:teuthology.orchestra.run.smithi091.stdout:Installed: 2019-12-22T22:07:37.272 INFO:teuthology.orchestra.run.smithi091.stdout: kernel.x86_64 0:3.10.0-1062.9.1.el7 2019-12-22T22:07:37.272 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:07:37.272 INFO:teuthology.orchestra.run.smithi091.stdout:Complete! 2019-12-22T22:07:37.321 INFO:teuthology.orchestra.run.smithi091:> rpm -q kernel --last | head -n 1 2019-12-22T22:07:37.725 INFO:teuthology.orchestra.run.smithi091.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Sun 22 Dec 2019 10:07:04 PM UTC 2019-12-22T22:07:37.731 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:07:37.731 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.4.1.el7.x86_64 Expected: 3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:07:37.732 INFO:teuthology.orchestra.run.smithi043:> uname -r 2019-12-22T22:07:37.829 INFO:teuthology.orchestra.run.smithi043.stdout:3.10.0-1062.4.1.el7.x86_64 2019-12-22T22:07:37.830 INFO:teuthology.task.kernel:Running kernel on smithi043: 3.10.0-1062.4.1.el7.x86_64 2019-12-22T22:07:37.830 INFO:teuthology.orchestra.run.smithi043:> sudo yum install -y kernel 2019-12-22T22:07:38.726 INFO:teuthology.orchestra.run.smithi043.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:07:38.862 INFO:teuthology.orchestra.run.smithi043.stdout:Determining fastest mirrors 2019-12-22T22:07:39.477 INFO:teuthology.orchestra.run.smithi043.stdout: * base: mirror.umd.edu 2019-12-22T22:07:39.493 INFO:teuthology.orchestra.run.smithi043.stdout: * epel: ftp.linux.ncsu.edu 2019-12-22T22:07:39.493 INFO:teuthology.orchestra.run.smithi043.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:07:39.493 INFO:teuthology.orchestra.run.smithi043.stdout: * updates: mirror.wdc1.us.leaseweb.net 2019-12-22T22:07:42.731 INFO:teuthology.orchestra.run.smithi043.stdout:Resolving Dependencies 2019-12-22T22:07:42.732 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:07:42.732 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package kernel.x86_64 0:3.10.0-1062.9.1.el7 will be installed 2019-12-22T22:07:43.990 INFO:teuthology.orchestra.run.smithi043.stdout:--> Finished Dependency Resolution 2019-12-22T22:07:44.822 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:07:44.823 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies Resolved 2019-12-22T22:07:44.824 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:07:44.824 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:07:44.825 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2019-12-22T22:07:44.825 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:07:44.825 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2019-12-22T22:07:44.825 INFO:teuthology.orchestra.run.smithi043.stdout: kernel x86_64 3.10.0-1062.9.1.el7 updates 50 M 2019-12-22T22:07:44.826 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:07:44.826 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2019-12-22T22:07:44.826 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:07:44.826 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2019-12-22T22:07:44.826 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:07:44.826 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 50 M 2019-12-22T22:07:44.826 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 64 M 2019-12-22T22:07:44.827 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading packages: 2019-12-22T22:07:44.830 INFO:teuthology.orchestra.run.smithi043.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-22T22:07:45.685 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2019-12-22T22:07:45.879 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2019-12-22T22:07:46.777 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded 2019-12-22T22:07:46.778 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2019-12-22T22:08:23.718 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-22T22:08:24.370 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-22T22:08:24.370 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:08:24.370 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2019-12-22T22:08:24.370 INFO:teuthology.orchestra.run.smithi043.stdout: kernel.x86_64 0:3.10.0-1062.9.1.el7 2019-12-22T22:08:24.370 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:08:24.371 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2019-12-22T22:08:24.415 INFO:teuthology.orchestra.run.smithi043:> rpm -q kernel --last | head -n 1 2019-12-22T22:08:24.829 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Sun 22 Dec 2019 10:07:49 PM UTC 2019-12-22T22:08:24.830 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:08:24.830 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.4.1.el7.x86_64 Expected: 3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:08:24.830 INFO:teuthology.orchestra.run.smithi107:> uname -r 2019-12-22T22:08:24.963 INFO:teuthology.orchestra.run.smithi107.stdout:3.10.0-1062.4.1.el7.x86_64 2019-12-22T22:08:24.964 INFO:teuthology.task.kernel:Running kernel on smithi107: 3.10.0-1062.4.1.el7.x86_64 2019-12-22T22:08:24.964 INFO:teuthology.orchestra.run.smithi107:> sudo yum install -y kernel 2019-12-22T22:08:25.933 INFO:teuthology.orchestra.run.smithi107.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:08:26.066 INFO:teuthology.orchestra.run.smithi107.stdout:Determining fastest mirrors 2019-12-22T22:08:26.747 INFO:teuthology.orchestra.run.smithi107.stdout: * base: repos-va.psychz.net 2019-12-22T22:08:26.754 INFO:teuthology.orchestra.run.smithi107.stdout: * epel: ftp.linux.ncsu.edu 2019-12-22T22:08:26.755 INFO:teuthology.orchestra.run.smithi107.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:08:26.755 INFO:teuthology.orchestra.run.smithi107.stdout: * updates: mirror.wdc1.us.leaseweb.net 2019-12-22T22:08:30.048 INFO:teuthology.orchestra.run.smithi107.stdout:Resolving Dependencies 2019-12-22T22:08:30.049 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:08:30.049 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package kernel.x86_64 0:3.10.0-1062.9.1.el7 will be installed 2019-12-22T22:08:31.438 INFO:teuthology.orchestra.run.smithi107.stdout:--> Finished Dependency Resolution 2019-12-22T22:08:32.320 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:08:32.320 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies Resolved 2019-12-22T22:08:32.330 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:08:32.330 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:08:32.330 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2019-12-22T22:08:32.330 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:08:32.330 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2019-12-22T22:08:32.331 INFO:teuthology.orchestra.run.smithi107.stdout: kernel x86_64 3.10.0-1062.9.1.el7 updates 50 M 2019-12-22T22:08:32.331 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:08:32.331 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2019-12-22T22:08:32.331 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:08:32.331 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2019-12-22T22:08:32.331 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:08:32.332 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 50 M 2019-12-22T22:08:32.332 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 64 M 2019-12-22T22:08:32.332 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading packages: 2019-12-22T22:08:32.337 INFO:teuthology.orchestra.run.smithi107.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-22T22:08:33.350 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2019-12-22T22:08:33.550 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2019-12-22T22:08:34.471 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded 2019-12-22T22:08:34.471 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2019-12-22T22:09:09.706 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-22T22:09:10.257 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-22T22:09:10.257 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:09:10.257 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2019-12-22T22:09:10.257 INFO:teuthology.orchestra.run.smithi107.stdout: kernel.x86_64 0:3.10.0-1062.9.1.el7 2019-12-22T22:09:10.257 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:09:10.257 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2019-12-22T22:09:10.302 INFO:teuthology.orchestra.run.smithi107:> rpm -q kernel --last | head -n 1 2019-12-22T22:09:10.704 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Sun 22 Dec 2019 10:08:37 PM UTC 2019-12-22T22:09:10.706 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:09:10.706 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.4.1.el7.x86_64 Expected: 3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:09:10.706 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2019-12-22T22:09:10.707 DEBUG:teuthology.task.kernel:src is distro, skipping download 2019-12-22T22:09:10.707 DEBUG:teuthology.task.kernel:src is distro, skipping download 2019-12-22T22:09:10.707 DEBUG:teuthology.task.kernel:src is distro, skipping download 2019-12-22T22:09:10.707 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2019-12-22T22:09:10.707 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi091.front.sepia.ceph.com, path=None, version=distro) 2019-12-22T22:09:10.707 INFO:teuthology.orchestra.run.smithi091:> rpm -q kernel --last | head -n 1 2019-12-22T22:09:11.094 INFO:teuthology.orchestra.run.smithi091.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Sun 22 Dec 2019 10:07:04 PM UTC 2019-12-22T22:09:11.095 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:09:11.095 INFO:teuthology.orchestra.run.smithi091:> sudo rpm -qi grub2-tools 2019-12-22T22:09:11.254 INFO:teuthology.orchestra.run.smithi091.stdout:Name : grub2-tools 2019-12-22T22:09:11.254 INFO:teuthology.orchestra.run.smithi091.stdout:Epoch : 1 2019-12-22T22:09:11.254 INFO:teuthology.orchestra.run.smithi091.stdout:Version : 2.02 2019-12-22T22:09:11.254 INFO:teuthology.orchestra.run.smithi091.stdout:Release : 0.76.el7.centos 2019-12-22T22:09:11.254 INFO:teuthology.orchestra.run.smithi091.stdout:Architecture: x86_64 2019-12-22T22:09:11.254 INFO:teuthology.orchestra.run.smithi091.stdout:Install Date: Tue 05 Nov 2019 03:40:24 PM UTC 2019-12-22T22:09:11.255 INFO:teuthology.orchestra.run.smithi091.stdout:Group : System Environment/Base 2019-12-22T22:09:11.255 INFO:teuthology.orchestra.run.smithi091.stdout:Size : 10080361 2019-12-22T22:09:11.255 INFO:teuthology.orchestra.run.smithi091.stdout:License : GPLv3+ 2019-12-22T22:09:11.255 INFO:teuthology.orchestra.run.smithi091.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:11 PM UTC, Key ID 24c6a8a7f4a80eb5 2019-12-22T22:09:11.255 INFO:teuthology.orchestra.run.smithi091.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2019-12-22T22:09:11.255 INFO:teuthology.orchestra.run.smithi091.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2019-12-22T22:09:11.255 INFO:teuthology.orchestra.run.smithi091.stdout:Build Host : kbuilder.bsys.centos.org 2019-12-22T22:09:11.256 INFO:teuthology.orchestra.run.smithi091.stdout:Relocations : (not relocatable) 2019-12-22T22:09:11.256 INFO:teuthology.orchestra.run.smithi091.stdout:Packager : CentOS BuildSystem 2019-12-22T22:09:11.256 INFO:teuthology.orchestra.run.smithi091.stdout:Vendor : CentOS 2019-12-22T22:09:11.256 INFO:teuthology.orchestra.run.smithi091.stdout:URL : http://www.gnu.org/software/grub/ 2019-12-22T22:09:11.256 INFO:teuthology.orchestra.run.smithi091.stdout:Summary : Support tools for GRUB. 2019-12-22T22:09:11.257 INFO:teuthology.orchestra.run.smithi091.stdout:Description : 2019-12-22T22:09:11.257 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:09:11.257 INFO:teuthology.orchestra.run.smithi091.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2019-12-22T22:09:11.258 INFO:teuthology.orchestra.run.smithi091.stdout:customizable bootloader with modular architecture. It supports a rich 2019-12-22T22:09:11.258 INFO:teuthology.orchestra.run.smithi091.stdout:variety of kernel formats, file systems, computer architectures and 2019-12-22T22:09:11.258 INFO:teuthology.orchestra.run.smithi091.stdout:hardware devices. 2019-12-22T22:09:11.258 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:09:11.259 INFO:teuthology.orchestra.run.smithi091.stdout:This subpackage provides tools for support of all platforms. 2019-12-22T22:09:11.260 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2019-12-22T22:09:11.261 INFO:teuthology.task.kernel:Updating grub on smithi091 to boot 3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:09:11.261 INFO:teuthology.orchestra.run.smithi091:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2019-12-22T22:09:11.757 INFO:teuthology.orchestra.run.smithi091.stderr:Generating grub configuration file ... 2019-12-22T22:09:11.890 INFO:teuthology.orchestra.run.smithi091.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:09:11.905 INFO:teuthology.orchestra.run.smithi091.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.9.1.el7.x86_64.img 2019-12-22T22:09:12.179 INFO:teuthology.orchestra.run.smithi091.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.1.el7.x86_64 2019-12-22T22:09:12.201 INFO:teuthology.orchestra.run.smithi091.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.1.el7.x86_64.img 2019-12-22T22:09:12.235 INFO:teuthology.orchestra.run.smithi091.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2019-12-22T22:09:12.252 INFO:teuthology.orchestra.run.smithi091.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2019-12-22T22:09:12.281 INFO:teuthology.orchestra.run.smithi091.stderr:Found linux image: /boot/vmlinuz-0-rescue-a1739c9c60b14f0891126d2e603bb003 2019-12-22T22:09:12.290 INFO:teuthology.orchestra.run.smithi091.stderr:Found initrd image: /boot/initramfs-0-rescue-a1739c9c60b14f0891126d2e603bb003.img 2019-12-22T22:09:13.424 INFO:teuthology.orchestra.run.smithi091.stderr:done 2019-12-22T22:09:13.425 INFO:teuthology.orchestra.run.smithi091:> mktemp 2019-12-22T22:09:13.537 INFO:teuthology.orchestra.run.smithi091.stdout:/tmp/tmp.RLFSsbvqor 2019-12-22T22:09:13.537 INFO:teuthology.orchestra.run.smithi091:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.RLFSsbvqor 2019-12-22T22:09:13.616 INFO:teuthology.orchestra.run.smithi091:> sudo chmod 0666 /tmp/tmp.RLFSsbvqor 2019-12-22T22:09:13.787 DEBUG:teuthology.orchestra.remote:smithi091:/tmp/tmp.RLFSsbvqor is 6KB 2019-12-22T22:09:13.854 INFO:teuthology.orchestra.run.smithi091:> rm -fr /tmp/tmp.RLFSsbvqor 2019-12-22T22:09:13.914 INFO:teuthology.orchestra.run.smithi091:> sudo grub2-set-default 0 2019-12-22T22:09:14.177 INFO:teuthology.orchestra.run.smithi091:> sudo shutdown -r now 2019-12-22T22:09:14.181 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2019-12-22T22:09:14.181 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi043.front.sepia.ceph.com, path=None, version=distro) 2019-12-22T22:09:14.181 INFO:teuthology.orchestra.run.smithi043:> rpm -q kernel --last | head -n 1 2019-12-22T22:09:14.538 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Sun 22 Dec 2019 10:07:49 PM UTC 2019-12-22T22:09:14.540 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:09:14.540 INFO:teuthology.orchestra.run.smithi043:> sudo rpm -qi grub2-tools 2019-12-22T22:09:14.718 INFO:teuthology.orchestra.run.smithi043.stdout:Name : grub2-tools 2019-12-22T22:09:14.718 INFO:teuthology.orchestra.run.smithi043.stdout:Epoch : 1 2019-12-22T22:09:14.718 INFO:teuthology.orchestra.run.smithi043.stdout:Version : 2.02 2019-12-22T22:09:14.718 INFO:teuthology.orchestra.run.smithi043.stdout:Release : 0.76.el7.centos 2019-12-22T22:09:14.718 INFO:teuthology.orchestra.run.smithi043.stdout:Architecture: x86_64 2019-12-22T22:09:14.719 INFO:teuthology.orchestra.run.smithi043.stdout:Install Date: Tue 05 Nov 2019 03:40:24 PM UTC 2019-12-22T22:09:14.719 INFO:teuthology.orchestra.run.smithi043.stdout:Group : System Environment/Base 2019-12-22T22:09:14.719 INFO:teuthology.orchestra.run.smithi043.stdout:Size : 10080361 2019-12-22T22:09:14.719 INFO:teuthology.orchestra.run.smithi043.stdout:License : GPLv3+ 2019-12-22T22:09:14.719 INFO:teuthology.orchestra.run.smithi043.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:11 PM UTC, Key ID 24c6a8a7f4a80eb5 2019-12-22T22:09:14.719 INFO:teuthology.orchestra.run.smithi043.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2019-12-22T22:09:14.719 INFO:teuthology.orchestra.run.smithi043.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2019-12-22T22:09:14.720 INFO:teuthology.orchestra.run.smithi043.stdout:Build Host : kbuilder.bsys.centos.org 2019-12-22T22:09:14.720 INFO:teuthology.orchestra.run.smithi043.stdout:Relocations : (not relocatable) 2019-12-22T22:09:14.720 INFO:teuthology.orchestra.run.smithi043.stdout:Packager : CentOS BuildSystem 2019-12-22T22:09:14.720 INFO:teuthology.orchestra.run.smithi043.stdout:Vendor : CentOS 2019-12-22T22:09:14.720 INFO:teuthology.orchestra.run.smithi043.stdout:URL : http://www.gnu.org/software/grub/ 2019-12-22T22:09:14.720 INFO:teuthology.orchestra.run.smithi043.stdout:Summary : Support tools for GRUB. 2019-12-22T22:09:14.720 INFO:teuthology.orchestra.run.smithi043.stdout:Description : 2019-12-22T22:09:14.721 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:09:14.721 INFO:teuthology.orchestra.run.smithi043.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2019-12-22T22:09:14.721 INFO:teuthology.orchestra.run.smithi043.stdout:customizable bootloader with modular architecture. It supports a rich 2019-12-22T22:09:14.721 INFO:teuthology.orchestra.run.smithi043.stdout:variety of kernel formats, file systems, computer architectures and 2019-12-22T22:09:14.721 INFO:teuthology.orchestra.run.smithi043.stdout:hardware devices. 2019-12-22T22:09:14.721 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:09:14.721 INFO:teuthology.orchestra.run.smithi043.stdout:This subpackage provides tools for support of all platforms. 2019-12-22T22:09:14.722 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2019-12-22T22:09:14.722 INFO:teuthology.task.kernel:Updating grub on smithi043 to boot 3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:09:14.723 INFO:teuthology.orchestra.run.smithi043:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2019-12-22T22:09:15.335 INFO:teuthology.orchestra.run.smithi043.stderr:Generating grub configuration file ... 2019-12-22T22:09:15.484 INFO:teuthology.orchestra.run.smithi043.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:09:15.497 INFO:teuthology.orchestra.run.smithi043.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.9.1.el7.x86_64.img 2019-12-22T22:09:15.788 INFO:teuthology.orchestra.run.smithi043.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.1.el7.x86_64 2019-12-22T22:09:15.813 INFO:teuthology.orchestra.run.smithi043.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.1.el7.x86_64.img 2019-12-22T22:09:15.852 INFO:teuthology.orchestra.run.smithi043.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2019-12-22T22:09:15.881 INFO:teuthology.orchestra.run.smithi043.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2019-12-22T22:09:15.917 INFO:teuthology.orchestra.run.smithi043.stderr:Found linux image: /boot/vmlinuz-0-rescue-a1739c9c60b14f0891126d2e603bb003 2019-12-22T22:09:15.929 INFO:teuthology.orchestra.run.smithi043.stderr:Found initrd image: /boot/initramfs-0-rescue-a1739c9c60b14f0891126d2e603bb003.img 2019-12-22T22:09:16.813 INFO:teuthology.orchestra.run.smithi043.stderr:done 2019-12-22T22:09:16.815 INFO:teuthology.orchestra.run.smithi043:> mktemp 2019-12-22T22:09:16.895 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.ayUsEkNFVb 2019-12-22T22:09:16.895 INFO:teuthology.orchestra.run.smithi043:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ayUsEkNFVb 2019-12-22T22:09:16.984 INFO:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.ayUsEkNFVb 2019-12-22T22:09:17.155 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.ayUsEkNFVb is 6KB 2019-12-22T22:09:17.235 INFO:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.ayUsEkNFVb 2019-12-22T22:09:17.301 INFO:teuthology.orchestra.run.smithi043:> sudo grub2-set-default 0 2019-12-22T22:09:17.473 INFO:teuthology.orchestra.run.smithi043:> sudo shutdown -r now 2019-12-22T22:09:17.476 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2019-12-22T22:09:17.477 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi107.front.sepia.ceph.com, path=None, version=distro) 2019-12-22T22:09:17.477 INFO:teuthology.orchestra.run.smithi107:> rpm -q kernel --last | head -n 1 2019-12-22T22:09:17.839 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Sun 22 Dec 2019 10:08:37 PM UTC 2019-12-22T22:09:17.841 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:09:17.841 INFO:teuthology.orchestra.run.smithi107:> sudo rpm -qi grub2-tools 2019-12-22T22:09:17.959 INFO:teuthology.orchestra.run.smithi107.stdout:Name : grub2-tools 2019-12-22T22:09:17.959 INFO:teuthology.orchestra.run.smithi107.stdout:Epoch : 1 2019-12-22T22:09:17.959 INFO:teuthology.orchestra.run.smithi107.stdout:Version : 2.02 2019-12-22T22:09:17.959 INFO:teuthology.orchestra.run.smithi107.stdout:Release : 0.76.el7.centos 2019-12-22T22:09:17.960 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: x86_64 2019-12-22T22:09:17.960 INFO:teuthology.orchestra.run.smithi107.stdout:Install Date: Tue 05 Nov 2019 03:40:24 PM UTC 2019-12-22T22:09:17.960 INFO:teuthology.orchestra.run.smithi107.stdout:Group : System Environment/Base 2019-12-22T22:09:17.960 INFO:teuthology.orchestra.run.smithi107.stdout:Size : 10080361 2019-12-22T22:09:17.960 INFO:teuthology.orchestra.run.smithi107.stdout:License : GPLv3+ 2019-12-22T22:09:17.960 INFO:teuthology.orchestra.run.smithi107.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:11 PM UTC, Key ID 24c6a8a7f4a80eb5 2019-12-22T22:09:17.960 INFO:teuthology.orchestra.run.smithi107.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2019-12-22T22:09:17.961 INFO:teuthology.orchestra.run.smithi107.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2019-12-22T22:09:17.961 INFO:teuthology.orchestra.run.smithi107.stdout:Build Host : kbuilder.bsys.centos.org 2019-12-22T22:09:17.961 INFO:teuthology.orchestra.run.smithi107.stdout:Relocations : (not relocatable) 2019-12-22T22:09:17.961 INFO:teuthology.orchestra.run.smithi107.stdout:Packager : CentOS BuildSystem 2019-12-22T22:09:17.961 INFO:teuthology.orchestra.run.smithi107.stdout:Vendor : CentOS 2019-12-22T22:09:17.961 INFO:teuthology.orchestra.run.smithi107.stdout:URL : http://www.gnu.org/software/grub/ 2019-12-22T22:09:17.961 INFO:teuthology.orchestra.run.smithi107.stdout:Summary : Support tools for GRUB. 2019-12-22T22:09:17.962 INFO:teuthology.orchestra.run.smithi107.stdout:Description : 2019-12-22T22:09:17.962 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:09:17.962 INFO:teuthology.orchestra.run.smithi107.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2019-12-22T22:09:17.962 INFO:teuthology.orchestra.run.smithi107.stdout:customizable bootloader with modular architecture. It supports a rich 2019-12-22T22:09:17.962 INFO:teuthology.orchestra.run.smithi107.stdout:variety of kernel formats, file systems, computer architectures and 2019-12-22T22:09:17.962 INFO:teuthology.orchestra.run.smithi107.stdout:hardware devices. 2019-12-22T22:09:17.962 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:09:17.963 INFO:teuthology.orchestra.run.smithi107.stdout:This subpackage provides tools for support of all platforms. 2019-12-22T22:09:17.963 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2019-12-22T22:09:17.964 INFO:teuthology.task.kernel:Updating grub on smithi107 to boot 3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:09:17.964 INFO:teuthology.orchestra.run.smithi107:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2019-12-22T22:09:18.408 INFO:teuthology.orchestra.run.smithi107.stderr:Generating grub configuration file ... 2019-12-22T22:09:18.536 INFO:teuthology.orchestra.run.smithi107.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:09:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.9.1.el7.x86_64.img 2019-12-22T22:09:18.791 INFO:teuthology.orchestra.run.smithi107.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.1.el7.x86_64 2019-12-22T22:09:18.817 INFO:teuthology.orchestra.run.smithi107.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.1.el7.x86_64.img 2019-12-22T22:09:18.850 INFO:teuthology.orchestra.run.smithi107.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2019-12-22T22:09:18.870 INFO:teuthology.orchestra.run.smithi107.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2019-12-22T22:09:18.897 INFO:teuthology.orchestra.run.smithi107.stderr:Found linux image: /boot/vmlinuz-0-rescue-a1739c9c60b14f0891126d2e603bb003 2019-12-22T22:09:18.906 INFO:teuthology.orchestra.run.smithi107.stderr:Found initrd image: /boot/initramfs-0-rescue-a1739c9c60b14f0891126d2e603bb003.img 2019-12-22T22:09:20.078 INFO:teuthology.orchestra.run.smithi107.stderr:done 2019-12-22T22:09:20.080 INFO:teuthology.orchestra.run.smithi107:> mktemp 2019-12-22T22:09:20.136 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.x34UtXmgdR 2019-12-22T22:09:20.136 INFO:teuthology.orchestra.run.smithi107:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.x34UtXmgdR 2019-12-22T22:09:20.202 INFO:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.x34UtXmgdR 2019-12-22T22:09:20.319 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.x34UtXmgdR is 6KB 2019-12-22T22:09:20.370 INFO:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.x34UtXmgdR 2019-12-22T22:09:20.415 INFO:teuthology.orchestra.run.smithi107:> sudo grub2-set-default 0 2019-12-22T22:09:20.653 INFO:teuthology.orchestra.run.smithi107:> sudo shutdown -r now 2019-12-22T22:09:20.657 INFO:teuthology.misc:Re-opening connections... 2019-12-22T22:09:20.657 INFO:teuthology.misc:trying to connect to ubuntu@smithi091.front.sepia.ceph.com 2019-12-22T22:09:20.657 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-22T22:09:20.657 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi091.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:10:20.659 DEBUG:teuthology.orchestra.remote:timed out 2019-12-22T22:10:20.662 INFO:teuthology.misc:trying to connect to ubuntu@smithi043.front.sepia.ceph.com 2019-12-22T22:10:20.662 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-22T22:10:20.663 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi043.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:11:20.665 DEBUG:teuthology.orchestra.remote:timed out 2019-12-22T22:11:20.666 INFO:teuthology.misc:trying to connect to ubuntu@smithi107.front.sepia.ceph.com 2019-12-22T22:11:20.666 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-22T22:11:20.667 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi107.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:11:21.918 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2019-12-22T22:11:21.918 DEBUG:teuthology.misc:waited 121.261660099 2019-12-22T22:11:22.920 INFO:teuthology.misc:trying to connect to ubuntu@smithi091.front.sepia.ceph.com 2019-12-22T22:11:22.920 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-22T22:11:22.921 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi091.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:11:23.052 INFO:teuthology.orchestra.run.smithi091:> true 2019-12-22T22:11:23.481 INFO:teuthology.misc:trying to connect to ubuntu@smithi107.front.sepia.ceph.com 2019-12-22T22:11:23.481 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-22T22:11:23.482 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi107.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:11:23.600 INFO:teuthology.orchestra.run.smithi107:> true 2019-12-22T22:11:23.936 DEBUG:teuthology.misc:waited 123.279333115 2019-12-22T22:11:24.937 INFO:teuthology.misc:trying to connect to ubuntu@smithi043.front.sepia.ceph.com 2019-12-22T22:11:24.937 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-22T22:11:24.938 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi043.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:11:25.048 INFO:teuthology.orchestra.run.smithi043:> true 2019-12-22T22:11:25.503 DEBUG:teuthology.misc:waited 124.846633196 2019-12-22T22:11:26.504 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2019-12-22T22:11:26.504 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "3.10.0-1062.9.1.el7.x86_64"... 2019-12-22T22:11:26.504 INFO:teuthology.orchestra.run.smithi091:> uname -r 2019-12-22T22:11:26.608 INFO:teuthology.orchestra.run.smithi091.stdout:3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:11:26.608 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1062.9.1.el7.x86_64 vs 3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:11:26.609 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2019-12-22T22:11:26.609 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2019-12-22T22:11:26.609 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "3.10.0-1062.9.1.el7.x86_64"... 2019-12-22T22:11:26.609 INFO:teuthology.orchestra.run.smithi043:> uname -r 2019-12-22T22:11:26.706 INFO:teuthology.orchestra.run.smithi043.stdout:3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:11:26.706 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1062.9.1.el7.x86_64 vs 3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:11:26.707 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2019-12-22T22:11:26.707 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2019-12-22T22:11:26.707 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "3.10.0-1062.9.1.el7.x86_64"... 2019-12-22T22:11:26.707 INFO:teuthology.orchestra.run.smithi107:> uname -r 2019-12-22T22:11:26.761 INFO:teuthology.orchestra.run.smithi107.stdout:3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:11:26.761 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1062.9.1.el7.x86_64 vs 3.10.0-1062.9.1.el7.x86_64 2019-12-22T22:11:26.762 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2019-12-22T22:11:27.762 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2019-12-22T22:11:27.763 INFO:teuthology.orchestra.run.smithi091:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2019-12-22T22:11:27.901 INFO:teuthology.orchestra.run.smithi091.stdout:ttyS1 2019-12-22T22:11:27.920 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2019-12-22T22:11:27.926 INFO:teuthology.orchestra.run.smithi043:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2019-12-22T22:11:28.090 INFO:teuthology.orchestra.run.smithi043.stdout:ttyS1 2019-12-22T22:11:28.101 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2019-12-22T22:11:28.101 INFO:teuthology.orchestra.run.smithi107:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2019-12-22T22:11:28.170 INFO:teuthology.orchestra.run.smithi107.stdout:ttyS1 2019-12-22T22:11:28.189 INFO:teuthology.run_tasks:Running task internal.base... 2019-12-22T22:11:28.221 INFO:teuthology.task.internal:Creating test directory... 2019-12-22T22:11:28.222 INFO:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2019-12-22T22:11:28.242 INFO:teuthology.orchestra.run.smithi091:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2019-12-22T22:11:28.265 INFO:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2019-12-22T22:11:28.353 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2019-12-22T22:11:28.399 INFO:teuthology.run_tasks:Running task internal.archive... 2019-12-22T22:11:28.441 INFO:teuthology.task.internal:Creating archive directory... 2019-12-22T22:11:28.441 INFO:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2019-12-22T22:11:28.452 INFO:teuthology.orchestra.run.smithi091:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2019-12-22T22:11:28.458 INFO:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2019-12-22T22:11:28.554 INFO:teuthology.run_tasks:Running task internal.coredump... 2019-12-22T22:11:28.631 INFO:teuthology.task.internal:Enabling coredump saving... 2019-12-22T22:11:28.631 INFO:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-22T22:11:28.636 INFO:teuthology.orchestra.run.smithi091:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-22T22:11:28.646 INFO:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-22T22:11:28.702 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-22T22:11:28.748 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-22T22:11:28.751 INFO:teuthology.orchestra.run.smithi091.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-22T22:11:28.754 INFO:teuthology.run_tasks:Running task internal.sudo... 2019-12-22T22:11:28.875 INFO:teuthology.task.internal:Configuring sudo... 2019-12-22T22:11:28.876 INFO:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2019-12-22T22:11:28.883 INFO:teuthology.orchestra.run.smithi091:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2019-12-22T22:11:28.887 INFO:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2019-12-22T22:11:29.010 INFO:teuthology.run_tasks:Running task internal.syslog... 2019-12-22T22:11:29.046 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2019-12-22T22:11:29.047 INFO:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2019-12-22T22:11:29.052 INFO:teuthology.orchestra.run.smithi091:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2019-12-22T22:11:29.056 INFO:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2019-12-22T22:11:29.142 INFO:teuthology.orchestra.run.smithi091:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-22T22:11:29.249 INFO:teuthology.orchestra.run.smithi091:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-22T22:11:29.373 INFO:teuthology.orchestra.run.smithi091:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-22T22:11:29.469 INFO:teuthology.orchestra.run.smithi091:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-22T22:11:29.585 INFO:teuthology.orchestra.run.smithi091:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2019-12-22T22:11:29.673 INFO:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-22T22:11:29.784 INFO:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-22T22:11:29.913 INFO:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-22T22:11:30.018 INFO:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-22T22:11:30.131 INFO:teuthology.orchestra.run.smithi043:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2019-12-22T22:11:30.253 INFO:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-22T22:11:30.321 INFO:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-22T22:11:30.395 INFO:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-22T22:11:30.463 INFO:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-22T22:11:30.530 INFO:teuthology.orchestra.run.smithi107:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2019-12-22T22:11:30.583 INFO:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2019-12-22T22:11:30.587 INFO:teuthology.orchestra.run.smithi091:> sudo service rsyslog restart 2019-12-22T22:11:30.599 INFO:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2019-12-22T22:11:30.707 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2019-12-22T22:11:30.738 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2019-12-22T22:11:30.781 INFO:teuthology.orchestra.run.smithi091.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2019-12-22T22:11:30.809 INFO:teuthology.run_tasks:Running task internal.timer... 2019-12-22T22:11:30.837 INFO:teuthology.task.internal:Starting timer... 2019-12-22T22:11:30.837 INFO:teuthology.run_tasks:Running task pcp... 2019-12-22T22:11:30.875 INFO:teuthology.run_tasks:Running task selinux... 2019-12-22T22:11:30.888 INFO:teuthology.orchestra.run.smithi043:> sudo service auditd rotate 2019-12-22T22:11:31.058 INFO:teuthology.orchestra.run.smithi043.stdout:Rotating logs: [ OK ] 2019-12-22T22:11:31.061 INFO:teuthology.orchestra.run.smithi091:> sudo service auditd rotate 2019-12-22T22:11:31.211 INFO:teuthology.orchestra.run.smithi091.stdout:Rotating logs: [ OK ] 2019-12-22T22:11:31.214 INFO:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2019-12-22T22:11:31.318 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: [ OK ] 2019-12-22T22:11:31.319 DEBUG:teuthology.task.selinux:Getting current SELinux state 2019-12-22T22:11:31.320 INFO:teuthology.orchestra.run.smithi091:> /usr/sbin/getenforce 2019-12-22T22:11:31.416 INFO:teuthology.orchestra.run.smithi091.stdout:Permissive 2019-12-22T22:11:31.416 INFO:teuthology.orchestra.run.smithi043:> /usr/sbin/getenforce 2019-12-22T22:11:31.511 INFO:teuthology.orchestra.run.smithi043.stdout:Permissive 2019-12-22T22:11:31.568 INFO:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2019-12-22T22:11:31.654 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2019-12-22T22:11:31.655 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi091.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi107.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi043.front.sepia.ceph.com': 'permissive'} 2019-12-22T22:11:31.655 INFO:teuthology.orchestra.run.smithi091:> sudo grep 'avc: .*denied' /var/log/audit/audit.log | grep -v '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\)' 2019-12-22T22:11:31.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-22T22:11:31.776 INFO:teuthology.orchestra.run.smithi043:> sudo grep 'avc: .*denied' /var/log/audit/audit.log | grep -v '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\)' 2019-12-22T22:11:31.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-22T22:11:31.887 INFO:teuthology.orchestra.run.smithi107:> sudo grep 'avc: .*denied' /var/log/audit/audit.log | grep -v '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\)' 2019-12-22T22:11:31.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-22T22:11:31.983 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2019-12-22T22:11:31.983 INFO:teuthology.orchestra.run.smithi091:> sudo /usr/sbin/setenforce permissive 2019-12-22T22:11:32.108 INFO:teuthology.orchestra.run.smithi043:> sudo /usr/sbin/setenforce permissive 2019-12-22T22:11:32.231 INFO:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2019-12-22T22:11:32.360 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2019-12-22T22:11:32.391 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2019-12-22T22:11:32.391 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2019-12-22T22:11:32.415 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2019-12-22T22:11:32.424 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi091.front.sepia.ceph.com,smithi043.front.sepia.ceph.com,smithi107.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2019-12-22T22:11:32.809 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2019-12-22T22:11:34.046 INFO:teuthology.task.ansible.out: [WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2019-12-22T22:11:36.868 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2019-12-22T22:11:36.892 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:36 +0000 (0:00:00.067) 0:00:00.067 ******* 2019-12-22T22:11:36.903 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:36 +0000 (0:00:00.013) 0:00:00.081 ******* 2019-12-22T22:11:36.916 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:36 +0000 (0:00:00.013) 0:00:00.094 ******* 2019-12-22T22:11:40.186 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi107.front.sepia.ceph.com] 2019-12-22T22:11:40.249 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] 2019-12-22T22:11:40.324 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:40 +0000 (0:00:03.401) 0:00:03.495 ******* 2019-12-22T22:11:40.326 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] 2019-12-22T22:11:40.397 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:40 +0000 (0:00:00.080) 0:00:03.575 ******* 2019-12-22T22:11:40.440 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:40 +0000 (0:00:00.042) 0:00:03.618 ******* 2019-12-22T22:11:40.790 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-22T22:11:40.828 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:40 +0000 (0:00:00.384) 0:00:04.003 ******* 2019-12-22T22:11:40.833 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-22T22:11:40.836 INFO:teuthology.task.ansible.out: ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-22T22:11:40.881 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:40 +0000 (0:00:00.055) 0:00:04.059 ******* 2019-12-22T22:11:40.928 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:40 +0000 (0:00:00.032) 0:00:04.092 ******* 2019-12-22T22:11:41.414 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** ok: [smithi091.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2019-12-22T22:11:41.463 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:41 +0000 (0:00:00.545) 0:00:04.637 ******* 2019-12-22T22:11:41.470 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} ok: [smithi043.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2019-12-22T22:11:41.510 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:41 +0000 (0:00:00.050) 0:00:04.688 ******* 2019-12-22T22:11:41.545 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:41 +0000 (0:00:00.032) 0:00:04.720 ******* 2019-12-22T22:11:41.548 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:11:41.550 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:11:41.588 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:41 +0000 (0:00:00.044) 0:00:04.765 ******* 2019-12-22T22:11:41.625 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:41 +0000 (0:00:00.034) 0:00:04.799 ******* 2019-12-22T22:11:41.633 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:11:41.667 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:41 +0000 (0:00:00.045) 0:00:04.844 ******* 2019-12-22T22:11:42.078 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2019-12-22T22:11:42.080 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} ok: [smithi107.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-22T22:11:42.081 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-22T22:11:42.119 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:42 +0000 (0:00:00.451) 0:00:05.296 ******* 2019-12-22T22:11:42.152 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:42 +0000 (0:00:00.033) 0:00:05.330 ******* 2019-12-22T22:11:42.187 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:42 +0000 (0:00:00.035) 0:00:05.365 ******* 2019-12-22T22:11:43.612 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2019-12-22T22:11:43.614 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1577052702.17-255056206294281/source", "state": "file", "uid": 0} 2019-12-22T22:11:43.615 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1577052702.21-250771346545878/source", "state": "file", "uid": 0} 2019-12-22T22:11:43.651 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:43 +0000 (0:00:01.464) 0:00:06.829 ******* 2019-12-22T22:11:43.689 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:43 +0000 (0:00:00.037) 0:00:06.867 ******* 2019-12-22T22:11:43.695 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1577052702.13-228112585798401/source", "state": "file", "uid": 0} 2019-12-22T22:11:43.734 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:43 +0000 (0:00:00.045) 0:00:06.912 ******* 2019-12-22T22:11:44.276 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2019-12-22T22:11:44.277 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:44.289 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:44.303 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:44.583 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:44.595 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:44.639 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:44.881 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:44.902 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:44.955 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:45.235 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:45.238 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:45.294 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:45.489 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:45.512 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:45.633 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:45.790 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:45.837 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:45.965 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:46.122 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:46.207 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:46.326 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-22T22:11:46.420 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2019-12-22T22:11:46.426 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2019-12-22T22:11:46.632 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:46 +0000 (0:00:02.889) 0:00:09.802 ******* 2019-12-22T22:11:46.657 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:46 +0000 (0:00:00.032) 0:00:09.834 ******* 2019-12-22T22:11:46.688 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:46 +0000 (0:00:00.030) 0:00:09.865 ******* 2019-12-22T22:11:47.590 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2019-12-22T22:11:47.591 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] 2019-12-22T22:11:47.603 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] 2019-12-22T22:11:47.678 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] 2019-12-22T22:11:47.722 INFO:teuthology.task.ansible.out: [WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2019-12-22T22:11:47.723 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:47 +0000 (0:00:01.034) 0:00:10.900 ******* 2019-12-22T22:11:47.758 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:47 +0000 (0:00:00.036) 0:00:10.936 ******* 2019-12-22T22:11:47.832 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:47 +0000 (0:00:00.073) 0:00:11.010 ******* 2019-12-22T22:11:48.349 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2019-12-22T22:11:48.350 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.210880", "end": "2019-12-22 22:11:48.331240", "rc": 0, "start": "2019-12-22 22:11:48.120360", "stderr": "", "stderr_lines": [], "stdout": "f4e10beea550ddcc5081090533acfa405cd36c43", "stdout_lines": ["f4e10beea550ddcc5081090533acfa405cd36c43"]} 2019-12-22T22:11:48.388 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:48 +0000 (0:00:00.556) 0:00:11.566 ******* 2019-12-22T22:11:48.395 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.227554", "end": "2019-12-22 22:11:48.345942", "rc": 0, "start": "2019-12-22 22:11:48.118388", "stderr": "", "stderr_lines": [], "stdout": "f4e10beea550ddcc5081090533acfa405cd36c43", "stdout_lines": ["f4e10beea550ddcc5081090533acfa405cd36c43"]} 2019-12-22T22:11:48.396 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.219809", "end": "2019-12-22 22:11:48.343970", "rc": 0, "start": "2019-12-22 22:11:48.124161", "stderr": "", "stderr_lines": [], "stdout": "f4e10beea550ddcc5081090533acfa405cd36c43", "stdout_lines": ["f4e10beea550ddcc5081090533acfa405cd36c43"]} 2019-12-22T22:11:48.433 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:48 +0000 (0:00:00.045) 0:00:11.611 ******* 2019-12-22T22:11:48.467 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:48 +0000 (0:00:00.033) 0:00:11.645 ******* 2019-12-22T22:11:48.745 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2019-12-22T22:11:48.746 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.013904", "end": "2019-12-22 22:11:48.709213", "failed_when_result": false, "rc": 0, "start": "2019-12-22 22:11:48.695309", "stderr": "", "stderr_lines": [], "stdout": "7b088106a1031364e6869fc2888a0e2542579008", "stdout_lines": ["7b088106a1031364e6869fc2888a0e2542579008"]} 2019-12-22T22:11:48.791 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:48 +0000 (0:00:00.322) 0:00:11.967 ******* 2019-12-22T22:11:48.808 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.019915", "end": "2019-12-22 22:11:48.747288", "failed_when_result": false, "rc": 0, "start": "2019-12-22 22:11:48.727373", "stderr": "", "stderr_lines": [], "stdout": "7b088106a1031364e6869fc2888a0e2542579008", "stdout_lines": ["7b088106a1031364e6869fc2888a0e2542579008"]} 2019-12-22T22:11:48.840 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:48 +0000 (0:00:00.051) 0:00:12.018 ******* 2019-12-22T22:11:48.846 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2019-12-22T22:11:48.848 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:11:48.884 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:48 +0000 (0:00:00.043) 0:00:12.062 ******* 2019-12-22T22:11:48.890 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:11:48.931 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:48 +0000 (0:00:00.045) 0:00:12.108 ******* 2019-12-22T22:11:48.936 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2019-12-22T22:11:48.937 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.010995", "end": "2019-12-22 22:11:48.878082", "failed_when_result": false, "rc": 0, "start": "2019-12-22 22:11:48.867087", "stderr": "", "stderr_lines": [], "stdout": "7b088106a1031364e6869fc2888a0e2542579008", "stdout_lines": ["7b088106a1031364e6869fc2888a0e2542579008"]} 2019-12-22T22:11:48.938 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:11:48.977 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:48 +0000 (0:00:00.047) 0:00:12.155 ******* 2019-12-22T22:11:49.013 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:49 +0000 (0:00:00.035) 0:00:12.191 ******* 2019-12-22T22:11:49.019 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:11:49.019 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2019-12-22T22:11:49.019 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:11:49.060 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:49 +0000 (0:00:00.044) 0:00:12.235 ******* 2019-12-22T22:11:49.094 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:49 +0000 (0:00:00.036) 0:00:12.272 ******* 2019-12-22T22:11:49.103 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2019-12-22T22:11:49.105 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Filter the managed_users list] *********************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:11:49.145 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:49 +0000 (0:00:00.050) 0:00:12.323 ******* 2019-12-22T22:11:49.179 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:49 +0000 (0:00:00.033) 0:00:12.356 ******* 2019-12-22T22:11:49.183 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2019-12-22T22:11:49.184 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:11:49.185 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:11:49.224 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:49 +0000 (0:00:00.046) 0:00:12.402 ******* 2019-12-22T22:11:49.258 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:49 +0000 (0:00:00.032) 0:00:12.435 ******* 2019-12-22T22:11:49.263 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2019-12-22T22:11:49.266 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:11:49.302 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:49 +0000 (0:00:00.044) 0:00:12.479 ******* 2019-12-22T22:11:49.308 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:11:49.345 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:49 +0000 (0:00:00.043) 0:00:12.522 ******* 2019-12-22T22:11:49.350 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2019-12-22T22:11:49.350 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:11:49.388 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:49 +0000 (0:00:00.043) 0:00:12.566 ******* 2019-12-22T22:11:49.405 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:11:49.433 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:49 +0000 (0:00:00.044) 0:00:12.610 ******* 2019-12-22T22:11:49.448 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:11:49.475 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:49 +0000 (0:00:00.042) 0:00:12.653 ******* 2019-12-22T22:11:49.501 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2019-12-22T22:11:49.503 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:11:49.539 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:49 +0000 (0:00:00.063) 0:00:12.716 ******* 2019-12-22T22:11:49.580 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* ok: [smithi043.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-22T22:11:49.612 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:49 +0000 (0:00:00.074) 0:00:12.790 ******* 2019-12-22T22:11:49.754 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-22T22:11:49.791 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:49 +0000 (0:00:00.178) 0:00:12.969 ******* 2019-12-22T22:11:49.872 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-22T22:11:49.909 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:11:49 +0000 (0:00:00.117) 0:00:13.087 ******* 2019-12-22T22:11:50.031 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2019-12-22T22:11:50.035 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2019-12-22T22:11:50.149 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2019-12-22T22:11:50.267 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2019-12-22T22:11:50.396 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2019-12-22T22:11:50.460 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2019-12-22T22:11:50.631 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2019-12-22T22:11:50.757 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2019-12-22T22:11:50.769 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2019-12-22T22:11:50.971 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2019-12-22T22:11:51.067 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/adeza", "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "move_home": false, "name": "adeza", "shell": "/bin/bash", "state": "present", "uid": 1104} 2019-12-22T22:11:51.090 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/adeza", "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "move_home": false, "name": "adeza", "shell": "/bin/bash", "state": "present", "uid": 1104} 2019-12-22T22:11:51.300 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/adeza", "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "move_home": false, "name": "adeza", "shell": "/bin/bash", "state": "present", "uid": 1104} 2019-12-22T22:11:51.379 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1105} 2019-12-22T22:11:51.452 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1105} 2019-12-22T22:11:51.643 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1105} 2019-12-22T22:11:51.683 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1106} 2019-12-22T22:11:51.793 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1106} 2019-12-22T22:11:51.998 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1106} 2019-12-22T22:11:52.032 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1107} 2019-12-22T22:11:52.128 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1107} 2019-12-22T22:11:52.347 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1107} 2019-12-22T22:11:52.349 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1108} 2019-12-22T22:11:52.479 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1108} 2019-12-22T22:11:52.692 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1108} 2019-12-22T22:11:52.703 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2019-12-22T22:11:52.817 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2019-12-22T22:11:53.011 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2019-12-22T22:11:53.038 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2019-12-22T22:11:53.165 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2019-12-22T22:11:53.340 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2019-12-22T22:11:53.341 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2019-12-22T22:11:53.521 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2019-12-22T22:11:53.692 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2019-12-22T22:11:53.694 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2019-12-22T22:11:53.928 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2019-12-22T22:11:54.043 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2019-12-22T22:11:54.074 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1114} 2019-12-22T22:11:54.282 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1114} 2019-12-22T22:11:54.314 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1114} 2019-12-22T22:11:54.452 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1115} 2019-12-22T22:11:54.630 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1115} 2019-12-22T22:11:54.664 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1115} 2019-12-22T22:11:54.803 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1116} 2019-12-22T22:11:54.941 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1116} 2019-12-22T22:11:54.984 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1116} 2019-12-22T22:11:55.136 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1117} 2019-12-22T22:11:55.275 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1117} 2019-12-22T22:11:55.309 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1117} 2019-12-22T22:11:55.500 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1118} 2019-12-22T22:11:55.626 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1118} 2019-12-22T22:11:55.659 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1118} 2019-12-22T22:11:55.836 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1119} 2019-12-22T22:11:55.979 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1119} 2019-12-22T22:11:56.016 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1119} 2019-12-22T22:11:56.150 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/gmeno", "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "move_home": false, "name": "gmeno", "shell": "/bin/bash", "state": "present", "uid": 1120} 2019-12-22T22:11:56.301 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/gmeno", "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "move_home": false, "name": "gmeno", "shell": "/bin/bash", "state": "present", "uid": 1120} 2019-12-22T22:11:56.340 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/gmeno", "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "move_home": false, "name": "gmeno", "shell": "/bin/bash", "state": "present", "uid": 1120} 2019-12-22T22:11:56.472 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1121} 2019-12-22T22:11:56.628 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1121} 2019-12-22T22:11:56.683 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1121} 2019-12-22T22:11:56.830 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1122} 2019-12-22T22:11:56.972 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1122} 2019-12-22T22:11:57.028 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1122} 2019-12-22T22:11:57.137 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1123} 2019-12-22T22:11:57.332 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1123} 2019-12-22T22:11:57.384 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1123} 2019-12-22T22:11:57.476 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1124} 2019-12-22T22:11:57.662 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1124} 2019-12-22T22:11:57.740 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1124} 2019-12-22T22:11:57.790 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1125} 2019-12-22T22:11:57.999 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1125} 2019-12-22T22:11:58.106 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1125} 2019-12-22T22:11:58.139 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1126} 2019-12-22T22:11:58.324 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1126} 2019-12-22T22:11:58.433 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1126} 2019-12-22T22:11:58.486 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1127} 2019-12-22T22:11:58.665 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1127} 2019-12-22T22:11:58.782 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1127} 2019-12-22T22:11:58.840 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1128} 2019-12-22T22:11:59.009 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1128} 2019-12-22T22:11:59.131 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1128} 2019-12-22T22:11:59.168 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1129} 2019-12-22T22:11:59.297 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1129} 2019-12-22T22:11:59.500 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1129} 2019-12-22T22:11:59.513 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1130} 2019-12-22T22:11:59.622 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1130} 2019-12-22T22:11:59.817 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1130} 2019-12-22T22:11:59.840 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1131} 2019-12-22T22:11:59.959 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1131} 2019-12-22T22:12:00.140 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1132} 2019-12-22T22:12:00.151 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1131} 2019-12-22T22:12:00.264 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1132} 2019-12-22T22:12:00.445 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1133} 2019-12-22T22:12:00.497 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1132} 2019-12-22T22:12:00.560 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1133} 2019-12-22T22:12:00.769 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1134} 2019-12-22T22:12:00.834 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1133} 2019-12-22T22:12:00.837 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1134} 2019-12-22T22:12:01.125 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1136} 2019-12-22T22:12:01.149 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1134} 2019-12-22T22:12:01.190 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1136} 2019-12-22T22:12:01.457 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1137} 2019-12-22T22:12:01.470 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1136} 2019-12-22T22:12:01.537 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1137} 2019-12-22T22:12:01.784 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1138} 2019-12-22T22:12:01.842 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1138} 2019-12-22T22:12:01.843 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1137} 2019-12-22T22:12:02.118 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1139} 2019-12-22T22:12:02.208 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1138} ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1139} 2019-12-22T22:12:02.489 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1140} 2019-12-22T22:12:02.524 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1140} 2019-12-22T22:12:02.536 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1139} 2019-12-22T22:12:02.788 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1141} 2019-12-22T22:12:02.831 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1141} 2019-12-22T22:12:02.874 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1140} 2019-12-22T22:12:03.107 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-22T22:12:03.108 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-22T22:12:03.204 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1141} 2019-12-22T22:12:03.418 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1143} 2019-12-22T22:12:03.419 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1143} 2019-12-22T22:12:03.557 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-22T22:12:03.771 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1144} 2019-12-22T22:12:03.781 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1144} 2019-12-22T22:12:03.922 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1143} 2019-12-22T22:12:04.149 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1146} 2019-12-22T22:12:04.286 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1144} 2019-12-22T22:12:04.324 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1146} 2019-12-22T22:12:04.325 INFO:teuthology.task.ansible.out: 2019-12-22T22:12:04.478 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1147} 2019-12-22T22:12:04.644 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1146} 2019-12-22T22:12:04.660 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1147} 2019-12-22T22:12:04.809 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1148} 2019-12-22T22:12:04.991 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1147} 2019-12-22T22:12:04.995 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1148} 2019-12-22T22:12:05.159 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1150} 2019-12-22T22:12:05.340 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1148} 2019-12-22T22:12:05.360 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1150} 2019-12-22T22:12:05.509 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1153} 2019-12-22T22:12:05.679 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1153} 2019-12-22T22:12:05.691 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1150} 2019-12-22T22:12:05.830 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-22T22:12:06.029 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-22T22:12:06.031 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1153} 2019-12-22T22:12:06.129 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1156} 2019-12-22T22:12:06.333 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-22T22:12:06.400 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1156} 2019-12-22T22:12:06.465 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1157} 2019-12-22T22:12:06.719 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1156} 2019-12-22T22:12:06.743 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1157} 2019-12-22T22:12:06.798 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1158} 2019-12-22T22:12:07.062 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1158} 2019-12-22T22:12:07.097 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1157} 2019-12-22T22:12:07.161 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1159} 2019-12-22T22:12:07.395 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1159} 2019-12-22T22:12:07.439 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1158} 2019-12-22T22:12:07.536 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1161} 2019-12-22T22:12:07.712 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1161} 2019-12-22T22:12:07.783 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1159} 2019-12-22T22:12:07.891 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1162} 2019-12-22T22:12:08.048 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1162} 2019-12-22T22:12:08.141 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1161} 2019-12-22T22:12:08.236 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1163} 2019-12-22T22:12:08.431 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1163} 2019-12-22T22:12:08.475 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1162} 2019-12-22T22:12:08.592 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1164} 2019-12-22T22:12:08.765 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1164} 2019-12-22T22:12:08.820 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1163} 2019-12-22T22:12:08.948 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1165} 2019-12-22T22:12:09.121 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1165} 2019-12-22T22:12:09.186 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1164} 2019-12-22T22:12:09.293 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1167} 2019-12-22T22:12:09.469 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1167} 2019-12-22T22:12:09.523 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1165} 2019-12-22T22:12:09.646 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1168} 2019-12-22T22:12:09.797 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1168} 2019-12-22T22:12:09.882 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1167} 2019-12-22T22:12:09.968 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1169} 2019-12-22T22:12:10.127 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1169} 2019-12-22T22:12:10.243 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1168} 2019-12-22T22:12:10.318 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1170} 2019-12-22T22:12:10.444 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1170} 2019-12-22T22:12:10.614 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1169} 2019-12-22T22:12:10.671 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1171} 2019-12-22T22:12:10.767 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1171} 2019-12-22T22:12:10.960 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1170} 2019-12-22T22:12:11.025 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1172} 2019-12-22T22:12:11.100 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1172} 2019-12-22T22:12:11.332 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1171} 2019-12-22T22:12:11.376 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1174} 2019-12-22T22:12:11.444 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1174} 2019-12-22T22:12:11.691 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1172} 2019-12-22T22:12:11.703 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-22T22:12:11.773 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-22T22:12:12.048 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1177} 2019-12-22T22:12:12.060 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1174} 2019-12-22T22:12:12.064 INFO:teuthology.task.ansible.out: 2019-12-22T22:12:12.105 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1177} 2019-12-22T22:12:12.391 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1178} 2019-12-22T22:12:12.425 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1178} 2019-12-22T22:12:12.435 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-22T22:12:12.737 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1179} 2019-12-22T22:12:12.749 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1179} 2019-12-22T22:12:12.804 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1177} 2019-12-22T22:12:13.101 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1180} 2019-12-22T22:12:13.114 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1180} 2019-12-22T22:12:13.166 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1178} 2019-12-22T22:12:13.444 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1181} 2019-12-22T22:12:13.456 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1181} 2019-12-22T22:12:13.520 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1179} 2019-12-22T22:12:13.776 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1182} 2019-12-22T22:12:13.809 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1182} 2019-12-22T22:12:13.863 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1180} 2019-12-22T22:12:14.125 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1184} 2019-12-22T22:12:14.137 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1184} 2019-12-22T22:12:14.236 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1181} 2019-12-22T22:12:14.464 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1185} 2019-12-22T22:12:14.492 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1185} 2019-12-22T22:12:14.604 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1182} 2019-12-22T22:12:14.804 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1186} 2019-12-22T22:12:14.838 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1186} 2019-12-22T22:12:14.948 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1184} 2019-12-22T22:12:15.128 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1187} 2019-12-22T22:12:15.171 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1187} 2019-12-22T22:12:15.276 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1185} 2019-12-22T22:12:15.476 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1188} 2019-12-22T22:12:15.510 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1188} 2019-12-22T22:12:15.606 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1186} 2019-12-22T22:12:15.779 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1189} 2019-12-22T22:12:15.845 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1189} 2019-12-22T22:12:15.980 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1187} 2019-12-22T22:12:16.172 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1190} 2019-12-22T22:12:16.205 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1190} 2019-12-22T22:12:16.335 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1188} 2019-12-22T22:12:16.473 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1191} 2019-12-22T22:12:16.560 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1191} 2019-12-22T22:12:16.692 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1189} 2019-12-22T22:12:16.799 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1192} 2019-12-22T22:12:16.935 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1192} 2019-12-22T22:12:17.047 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1190} 2019-12-22T22:12:17.144 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1193} 2019-12-22T22:12:17.270 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1193} 2019-12-22T22:12:17.390 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1191} 2019-12-22T22:12:17.442 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1194} 2019-12-22T22:12:17.611 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1194} 2019-12-22T22:12:17.762 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1195} 2019-12-22T22:12:17.781 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1192} 2019-12-22T22:12:17.976 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1195} 2019-12-22T22:12:18.098 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1196} 2019-12-22T22:12:18.132 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1193} 2019-12-22T22:12:18.283 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1196} 2019-12-22T22:12:18.445 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1197} 2019-12-22T22:12:18.465 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1194} 2019-12-22T22:12:18.623 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1197} 2019-12-22T22:12:18.809 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1198} 2019-12-22T22:12:18.867 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1195} 2019-12-22T22:12:18.972 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1198} 2019-12-22T22:12:19.197 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1199} 2019-12-22T22:12:19.241 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1196} 2019-12-22T22:12:19.344 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1199} 2019-12-22T22:12:19.534 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1200} 2019-12-22T22:12:19.567 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1197} 2019-12-22T22:12:19.687 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1200} 2019-12-22T22:12:19.867 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1201} 2019-12-22T22:12:19.933 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1198} 2019-12-22T22:12:19.999 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1201} 2019-12-22T22:12:20.213 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1202} 2019-12-22T22:12:20.276 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1199} 2019-12-22T22:12:20.388 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1202} 2019-12-22T22:12:20.548 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1203} 2019-12-22T22:12:20.617 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1200} 2019-12-22T22:12:20.725 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1203} 2019-12-22T22:12:20.971 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1201} 2019-12-22T22:12:20.997 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1204} 2019-12-22T22:12:21.242 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1204} 2019-12-22T22:12:21.317 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1205} 2019-12-22T22:12:21.331 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1202} 2019-12-22T22:12:21.597 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1205} 2019-12-22T22:12:21.663 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1206} 2019-12-22T22:12:21.676 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1203} 2019-12-22T22:12:21.945 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1206} 2019-12-22T22:12:22.019 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1207} 2019-12-22T22:12:22.151 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1204} 2019-12-22T22:12:22.318 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1207} 2019-12-22T22:12:22.325 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1208} 2019-12-22T22:12:22.498 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1205} 2019-12-22T22:12:22.640 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1208} 2019-12-22T22:12:22.673 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1209} 2019-12-22T22:12:22.853 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1206} 2019-12-22T22:12:22.982 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1209} 2019-12-22T22:12:23.016 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1210} 2019-12-22T22:12:23.200 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1207} 2019-12-22T22:12:23.366 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1211} ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1210} 2019-12-22T22:12:23.564 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1208} 2019-12-22T22:12:23.656 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1212} 2019-12-22T22:12:23.684 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1211} 2019-12-22T22:12:23.926 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1209} 2019-12-22T22:12:24.015 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1213} 2019-12-22T22:12:24.027 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1212} 2019-12-22T22:12:24.288 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1210} 2019-12-22T22:12:24.363 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1214} 2019-12-22T22:12:24.400 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1213} 2019-12-22T22:12:24.618 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1211} 2019-12-22T22:12:24.719 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-22T22:12:24.765 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1214} 2019-12-22T22:12:24.973 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1212} 2019-12-22T22:12:25.058 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10102, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 10102} 2019-12-22T22:12:25.102 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-22T22:12:25.325 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1213} 2019-12-22T22:12:25.453 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10102, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 10102} 2019-12-22T22:12:25.688 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1214} 2019-12-22T22:12:25.764 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "name": "rraja", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2019-12-22T22:12:26.041 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-22T22:12:26.115 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "name": "rraja", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2019-12-22T22:12:26.286 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f', u'name': u'toabctl'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "name": "toabctl", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2019-12-22T22:12:26.404 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10102, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 10102} 2019-12-22T22:12:26.748 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f', u'name': u'toabctl'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "name": "toabctl", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2019-12-22T22:12:26.953 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72', u'name': u'mgfritch'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72"}, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2019-12-22T22:12:27.083 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:12:27 +0000 (0:00:37.174) 0:00:50.261 ******* 2019-12-22T22:12:27.087 INFO:teuthology.task.ansible.out: 2019-12-22T22:12:27.091 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "name": "rraja", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2019-12-22T22:12:27.108 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2019-12-22T22:12:27.147 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:12:27 +0000 (0:00:00.060) 0:00:50.323 ******* 2019-12-22T22:12:27.198 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2019-12-22T22:12:27.201 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "tr 2019-12-22T22:12:27.203 INFO:teuthology.task.ansible.out:ociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7 2019-12-22T22:12:27.205 INFO:teuthology.task.ansible.out:L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32 2019-12-22T22:12:27.207 INFO:teuthology.task.ansible.out:f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b 2019-12-22T22:12:27.208 INFO:teuthology.task.ansible.out:09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZB 2019-12-22T22:12:27.209 INFO:teuthology.task.ansible.out:Jy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e 2019-12-22T22:12:27.210 INFO:teuthology.task.ansible.out:66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa 2019-12-22T22:12:27.214 INFO:teuthology.task.ansible.out:8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f 2019-12-22T22:12:27.215 INFO:teuthology.task.ansible.out:826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd1 2019-12-22T22:12:27.216 INFO:teuthology.task.ansible.out:7f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 2019-12-22T22:12:27.221 INFO:teuthology.task.ansible.out: 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn" 2019-12-22T22:12:27.229 INFO:teuthology.task.ansible.out:: "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72"}]}, "changed": false} 2019-12-22T22:12:27.256 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:12:27 +0000 (0:00:00.111) 0:00:50.434 ******* 2019-12-22T22:12:27.394 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2019-12-22T22:12:27.394 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72', u'name': u'mgfritch'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72"}, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2019-12-22T22:12:27.560 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:12:27 +0000 (0:00:00.303) 0:00:50.738 ******* 2019-12-22T22:12:27.600 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2019-12-22T22:12:27.660 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:12:27 +0000 (0:00:00.098) 0:00:50.836 ******* 2019-12-22T22:12:27.675 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2019-12-22T22:12:27.675 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f', u'name': u'toabctl'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "name": "toabctl", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2019-12-22T22:12:27.734 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2019-12-22T22:12:27.743 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "tr 2019-12-22T22:12:27.744 INFO:teuthology.task.ansible.out:ociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7 2019-12-22T22:12:27.746 INFO:teuthology.task.ansible.out:L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32 2019-12-22T22:12:27.747 INFO:teuthology.task.ansible.out:f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b 2019-12-22T22:12:27.749 INFO:teuthology.task.ansible.out:09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e 2019-12-22T22:12:27.752 INFO:teuthology.task.ansible.out:66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6c 2019-12-22T22:12:27.752 INFO:teuthology.task.ansible.out:GN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa 2019-12-22T22:12:27.753 INFO:teuthology.task.ansible.out:8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f 2019-12-22T22:12:27.755 INFO:teuthology.task.ansible.out:826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd1 2019-12-22T22:12:27.756 INFO:teuthology.task.ansible.out:7f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 2019-12-22T22:12:27.757 INFO:teuthology.task.ansible.out: 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn" 2019-12-22T22:12:27.759 INFO:teuthology.task.ansible.out:: "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72"}]}, "changed": false} 2019-12-22T22:12:27.799 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:12:27 +0000 (0:00:00.140) 0:00:50.976 ******* 2019-12-22T22:12:28.267 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2019-12-22T22:12:28.269 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72', u'name': u'mgfritch'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72"}, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2019-12-22T22:12:28.281 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2019-12-22T22:12:28.283 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com -> localhost] => {"after": "f4e10beea550ddcc5081090533acfa405cd36c43", "attempts": 1, "before": "f4e10beea550ddcc5081090533acfa405cd36c43", "changed": false, "remote_url_changed": false} 2019-12-22T22:12:28.350 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:12:28 +0000 (0:00:00.549) 0:00:51.526 ******* 2019-12-22T22:12:28.382 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2019-12-22T22:12:28.468 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:12:28 +0000 (0:00:00.119) 0:00:51.646 ******* 2019-12-22T22:12:28.500 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2019-12-22T22:12:28.539 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:12:28 +0000 (0:00:00.066) 0:00:51.712 ******* 2019-12-22T22:12:28.543 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2019-12-22T22:12:28.545 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com -> localhost] => {"after": "f4e10beea550ddcc5081090533acfa405cd36c43", "attempts": 1, "before": "f4e10beea550ddcc5081090533acfa405cd36c43", "changed": false, "remote_url_changed": false} 2019-12-22T22:12:28.589 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:12:28 +0000 (0:00:00.055) 0:00:51.767 ******* 2019-12-22T22:12:28.596 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2019-12-22T22:12:28.601 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "tr 2019-12-22T22:12:28.603 INFO:teuthology.task.ansible.out:ociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7 2019-12-22T22:12:28.604 INFO:teuthology.task.ansible.out:L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32 2019-12-22T22:12:28.606 INFO:teuthology.task.ansible.out:f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b 2019-12-22T22:12:28.607 INFO:teuthology.task.ansible.out:09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e 2019-12-22T22:12:28.609 INFO:teuthology.task.ansible.out:66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6c 2019-12-22T22:12:28.609 INFO:teuthology.task.ansible.out:GN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa 2019-12-22T22:12:28.610 INFO:teuthology.task.ansible.out:8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f 2019-12-22T22:12:28.612 INFO:teuthology.task.ansible.out:826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd1 2019-12-22T22:12:28.613 INFO:teuthology.task.ansible.out:7f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 2019-12-22T22:12:28.615 INFO:teuthology.task.ansible.out: 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn" 2019-12-22T22:12:28.616 INFO:teuthology.task.ansible.out:: "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72"}]}, "changed": false} 2019-12-22T22:12:28.657 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:12:28 +0000 (0:00:00.066) 0:00:51.833 ******* 2019-12-22T22:12:28.754 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2019-12-22T22:12:28.988 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2019-12-22T22:12:29.155 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2019-12-22T22:12:29.336 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2019-12-22T22:12:29.346 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com -> localhost] => {"after": "f4e10beea550ddcc5081090533acfa405cd36c43", "attempts": 1, "before": "f4e10beea550ddcc5081090533acfa405cd36c43", "changed": false, "remote_url_changed": false} 2019-12-22T22:12:29.380 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:12:29 +0000 (0:00:00.722) 0:00:52.556 ******* 2019-12-22T22:12:29.387 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2019-12-22T22:12:29.535 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2019-12-22T22:12:29.711 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2019-12-22T22:12:29.793 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2019-12-22T22:12:29.881 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCWy2i6Iv6IWLh53Z44O4fQ0ctxR8odCAWIZAaSQXAxxC++RS+95Zr+BZ5wJwsBOyP1G4AAhl+9uwnn3NmDDG7fKGexPoQrTZZ27KdJ3bHpZRq5LJf9byWFwz4rDsqIqkGjI9sUkfa6nV7ci2BdEkIgLupTT2uXUzTb/njOjI7Fc+cQ7ac4QfJ+yeIcqxeje/YYDF3068V4z1yynk9q2KRXYdERKQghaWDNJlbdBhn/fmsam6okOtUnsXjE88qGPd+LVHWfK0EfYvYXiAoOaJDK8d4EXnJ4RKJceQPBk8ravuQtKVwnRIr8WtmILHuoztthfwqb2WvZ5iMDt6v7oQZ alfredo@papaya.local", "key_options": null, "keyfile": "/home/adeza/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adeza", "validate_certs": true} 2019-12-22T22:12:30.049 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCWy2i6Iv6IWLh53Z44O4fQ0ctxR8odCAWIZAaSQXAxxC++RS+95Zr+BZ5wJwsBOyP1G4AAhl+9uwnn3NmDDG7fKGexPoQrTZZ27KdJ3bHpZRq5LJf9byWFwz4rDsqIqkGjI9sUkfa6nV7ci2BdEkIgLupTT2uXUzTb/njOjI7Fc+cQ7ac4QfJ+yeIcqxeje/YYDF3068V4z1yynk9q2KRXYdERKQghaWDNJlbdBhn/fmsam6okOtUnsXjE88qGPd+LVHWfK0EfYvYXiAoOaJDK8d4EXnJ4RKJceQPBk8ravuQtKVwnRIr8WtmILHuoztthfwqb2WvZ5iMDt6v7oQZ alfredo@papaya.local", "key_options": null, "keyfile": "/home/adeza/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adeza", "validate_certs": true} 2019-12-22T22:12:30.138 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2019-12-22T22:12:30.224 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "va 2019-12-22T22:12:30.226 INFO:teuthology.task.ansible.out:lidate_certs": true} 2019-12-22T22:12:30.395 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "va 2019-12-22T22:12:30.403 INFO:teuthology.task.ansible.out:lidate_certs": true} 2019-12-22T22:12:30.509 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2019-12-22T22:12:30.564 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2019-12-22T22:12:30.742 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2019-12-22T22:12:30.862 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCWy2i6Iv6IWLh53Z44O4fQ0ctxR8odCAWIZAaSQXAxxC++RS+95Zr+BZ5wJwsBOyP1G4AAhl+9uwnn3NmDDG7fKGexPoQrTZZ27KdJ3bHpZRq5LJf9byWFwz4rDsqIqkGjI9sUkfa6nV7ci2BdEkIgLupTT2uXUzTb/njOjI7Fc+cQ7ac4QfJ+yeIcqxeje/YYDF3068V4z1yynk9q2KRXYdERKQghaWDNJlbdBhn/fmsam6okOtUnsXjE88qGPd+LVHWfK0EfYvYXiAoOaJDK8d4EXnJ4RKJceQPBk8ravuQtKVwnRIr8WtmILHuoztthfwqb2WvZ5iMDt6v7oQZ alfredo@papaya.local", "key_options": null, "keyfile": "/home/adeza/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adeza", "validate_certs": true} 2019-12-22T22:12:30.930 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZtt 2019-12-22T22:12:30.938 INFO:teuthology.task.ansible.out:Edz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2019-12-22T22:12:31.121 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZtt 2019-12-22T22:12:31.122 INFO:teuthology.task.ansible.out:Edz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2019-12-22T22:12:31.228 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2019-12-22T22:12:31.238 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "va 2019-12-22T22:12:31.246 INFO:teuthology.task.ansible.out:lidate_certs": true} 2019-12-22T22:12:31.460 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2019-12-22T22:12:31.516 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2019-12-22T22:12:31.527 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:12:31.604 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2019-12-22T22:12:31.782 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2019-12-22T22:12:31.805 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:12:31.894 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uT 2019-12-22T22:12:31.900 INFO:teuthology.task.ansible.out:fMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2019-12-22T22:12:31.908 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:12:31.910 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:12:31.920 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:12:31.976 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZtt 2019-12-22T22:12:31.978 INFO:teuthology.task.ansible.out:Edz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2019-12-22T22:12:32.158 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uT 2019-12-22T22:12:32.159 INFO:teuthology.task.ansible.out:fMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2019-12-22T22:12:32.169 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:12:32.180 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:12:32.192 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:12:32.260 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2019-12-22T22:12:32.317 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2019-12-22T22:12:32.537 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2019-12-22T22:12:32.617 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+ 2019-12-22T22:12:32.625 INFO:teuthology.task.ansible.out:2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2019-12-22T22:12:32.656 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2019-12-22T22:12:32.695 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:12:32.914 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+ 2019-12-22T22:12:32.916 INFO:teuthology.task.ansible.out:2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2019-12-22T22:12:32.946 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2019-12-22T22:12:33.036 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uT 2019-12-22T22:12:33.038 INFO:teuthology.task.ansible.out:fMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2019-12-22T22:12:33.049 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:12:33.064 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:12:33.078 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:12:33.275 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2019-12-22T22:12:33.292 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2019-12-22T22:12:33.454 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2019-12-22T22:12:33.656 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvWnKfzTd574C4GXg1mr4pbJOInKv4qMRf0DwksTzqtXStlhT+PaO2C8p48U2Xjz5hmPK9J+iHA2xKwI6lKI194wUo3tW283euRyTCN7LlbV6tgD9k78hDaeA+53MHYmwbxjHEqAadqrLU+68LfmL7gaaUSToP/Ap+hdcDC7H9lulMcxkUeahEiDfG2s1mD8IUM6XVuqZM9WEARKpF1oSEBKhMMob9YJZPpJWipuAGkPvDA0yHK1lB58Yr2x0yaezVjQc7V2dwaY1gXR6AscU5WzlwWNNwXHqPdyyWRaGx/9Fk00yfzX4V7ZqTvKdMLKwB7EnamduTMPsaoKBUBJCh gmeno@gmeno-virtual-machine", "key_options": null, "keyfile": "/home/gmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gmeno", "validate_certs": true} 2019-12-22T22:12:33.794 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2019-12-22T22:12:33.950 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+ 2019-12-22T22:12:33.955 INFO:teuthology.task.ansible.out:2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2019-12-22T22:12:34.004 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2019-12-22T22:12:34.159 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvWnKfzTd574C4GXg1mr4pbJOInKv4qMRf0DwksTzqtXStlhT+PaO2C8p48U2Xjz5hmPK9J+iHA2xKwI6lKI194wUo3tW283euRyTCN7LlbV6tgD9k78hDaeA+53MHYmwbxjHEqAadqrLU+68LfmL7gaaUSToP/Ap+hdcDC7H9lulMcxkUeahEiDfG2s1mD8IUM6XVuqZM9WEARKpF1oSEBKhMMob9YJZPpJWipuAGkPvDA0yHK1lB58Yr2x0yaezVjQc7V2dwaY1gXR6AscU5WzlwWNNwXHqPdyyWRaGx/9Fk00yfzX4V7ZqTvKdMLKwB7EnamduTMPsaoKBUBJCh gmeno@gmeno-virtual-machine", "key_options": null, "keyfile": "/home/gmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gmeno", "validate_certs": true} 2019-12-22T22:12:34.311 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2019-12-22T22:12:34.480 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3 2019-12-22T22:12:34.482 INFO:teuthology.task.ansible.out:c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2019-12-22T22:12:34.655 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2019-12-22T22:12:34.816 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF 2019-12-22T22:12:34.818 INFO:teuthology.task.ansible.out: wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2019-12-22T22:12:34.832 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2019-12-22T22:12:35.010 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvWnKfzTd574C4GXg1mr4pbJOInKv4qMRf0DwksTzqtXStlhT+PaO2C8p48U2Xjz5hmPK9J+iHA2xKwI6lKI194wUo3tW283euRyTCN7LlbV6tgD9k78hDaeA+53MHYmwbxjHEqAadqrLU+68LfmL7gaaUSToP/Ap+hdcDC7H9lulMcxkUeahEiDfG2s1mD8IUM6XVuqZM9WEARKpF1oSEBKhMMob9YJZPpJWipuAGkPvDA0yHK1lB58Yr2x0yaezVjQc7V2dwaY1gXR6AscU5WzlwWNNwXHqPdyyWRaGx/9Fk00yfzX4V7ZqTvKdMLKwB7EnamduTMPsaoKBUBJCh gmeno@gmeno-virtual-machine", "key_options": null, "keyfile": "/home/gmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gmeno", "validate_certs": true} 2019-12-22T22:12:35.159 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1 2019-12-22T22:12:35.160 INFO:teuthology.task.ansible.out:Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2019-12-22T22:12:35.182 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2019-12-22T22:12:35.398 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2019-12-22T22:12:35.527 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkR 2019-12-22T22:12:35.528 INFO:teuthology.task.ansible.out:hoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2019-12-22T22:12:35.548 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1 2019-12-22T22:12:35.549 INFO:teuthology.task.ansible.out:Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2019-12-22T22:12:35.739 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2019-12-22T22:12:35.858 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2019-12-22T22:12:35.932 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkR 2019-12-22T22:12:35.934 INFO:teuthology.task.ansible.out:hoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2019-12-22T22:12:36.098 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2019-12-22T22:12:36.200 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2019-12-22T22:12:36.263 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2019-12-22T22:12:36.468 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1 2019-12-22T22:12:36.477 INFO:teuthology.task.ansible.out:Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2019-12-22T22:12:36.537 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwm 2019-12-22T22:12:36.541 INFO:teuthology.task.ansible.out:ih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2019-12-22T22:12:36.623 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2019-12-22T22:12:36.816 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkR 2019-12-22T22:12:36.818 INFO:teuthology.task.ansible.out:hoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2019-12-22T22:12:36.871 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2019-12-22T22:12:36.988 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwm 2019-12-22T22:12:36.989 INFO:teuthology.task.ansible.out:ih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2019-12-22T22:12:37.181 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2019-12-22T22:12:37.203 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2019-12-22T22:12:37.331 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2019-12-22T22:12:37.534 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2019-12-22T22:12:37.563 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2019-12-22T22:12:37.673 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2019-12-22T22:12:37.900 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwm 2019-12-22T22:12:37.902 INFO:teuthology.task.ansible.out:ih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2019-12-22T22:12:38.034 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2019-12-22T22:12:38.233 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2019-12-22T22:12:38.268 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2019-12-22T22:12:38.365 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2019-12-22T22:12:38.616 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2 2019-12-22T22:12:38.617 INFO:teuthology.task.ansible.out:CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2019-12-22T22:12:38.738 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2019-12-22T22:12:38.921 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2019-12-22T22:12:38.968 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2019-12-22T22:12:39.105 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2 2019-12-22T22:12:39.107 INFO:teuthology.task.ansible.out:CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2019-12-22T22:12:39.277 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2019-12-22T22:12:39.338 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2019-12-22T22:12:39.497 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2019-12-22T22:12:39.605 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2019-12-22T22:12:39.690 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2019-12-22T22:12:39.840 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2019-12-22T22:12:39.945 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2019-12-22T22:12:40.044 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2 2019-12-22T22:12:40.046 INFO:teuthology.task.ansible.out:CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2019-12-22T22:12:40.202 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2019-12-22T22:12:40.304 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2019-12-22T22:12:40.398 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2019-12-22T22:12:40.553 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2019-12-22T22:12:40.669 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2019-12-22T22:12:40.769 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2019-12-22T22:12:40.905 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2019-12-22T22:12:41.018 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-22T22:12:41.141 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2019-12-22T22:12:41.312 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2019-12-22T22:12:41.373 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2019-12-22T22:12:41.508 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2019-12-22T22:12:41.697 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-22T22:12:41.698 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2019-12-22T22:12:41.867 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2019-12-22T22:12:42.024 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6 2019-12-22T22:12:42.025 INFO:teuthology.task.ansible.out:AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2019-12-22T22:12:42.066 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2019-12-22T22:12:42.196 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2019-12-22T22:12:42.377 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2Ruq 2019-12-22T22:12:42.379 INFO:teuthology.task.ansible.out:iZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2019-12-22T22:12:42.423 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2019-12-22T22:12:42.555 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-22T22:12:42.732 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2019-12-22T22:12:42.797 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6 2019-12-22T22:12:42.798 INFO:teuthology.task.ansible.out:AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2019-12-22T22:12:42.895 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2019-12-22T22:12:43.050 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2019-12-22T22:12:43.174 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2Ruq 2019-12-22T22:12:43.176 INFO:teuthology.task.ansible.out:iZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2019-12-22T22:12:43.251 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2019-12-22T22:12:43.367 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2019-12-22T22:12:43.532 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2019-12-22T22:12:43.613 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6 2019-12-22T22:12:43.615 INFO:teuthology.task.ansible.out:AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2019-12-22T22:12:43.701 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options" 2019-12-22T22:12:43.702 INFO:teuthology.task.ansible.out:: null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-22T22:12:43.872 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2019-12-22T22:12:43.961 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2Ruq 2019-12-22T22:12:43.963 INFO:teuthology.task.ansible.out:iZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2019-12-22T22:12:44.037 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+z 2019-12-22T22:12:44.039 INFO:teuthology.task.ansible.out:mfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbB 2019-12-22T22:12:44.042 INFO:teuthology.task.ansible.out:mS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2019-12-22T22:12:44.255 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2019-12-22T22:12:44.342 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2019-12-22T22:12:44.432 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2019-12-22T22:12:44.630 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options" 2019-12-22T22:12:44.636 INFO:teuthology.task.ansible.out:: null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-22T22:12:44.698 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2019-12-22T22:12:44.752 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2019-12-22T22:12:45.005 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+z 2019-12-22T22:12:45.007 INFO:teuthology.task.ansible.out:mfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbB 2019-12-22T22:12:45.009 INFO:teuthology.task.ansible.out:mS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2019-12-22T22:12:45.072 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2019-12-22T22:12:45.096 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2019-12-22T22:12:45.351 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2019-12-22T22:12:45.425 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options" 2019-12-22T22:12:45.426 INFO:teuthology.task.ansible.out:: null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-22T22:12:45.469 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2019-12-22T22:12:45.709 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2019-12-22T22:12:45.795 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+z 2019-12-22T22:12:45.797 INFO:teuthology.task.ansible.out:mfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbB 2019-12-22T22:12:45.798 INFO:teuthology.task.ansible.out:mS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2019-12-22T22:12:45.807 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2019-12-22T22:12:46.048 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2019-12-22T22:12:46.150 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2019-12-22T22:12:46.179 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2019-12-22T22:12:46.419 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2019-12-22T22:12:46.495 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2019-12-22T22:12:46.549 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2019-12-22T22:12:46.776 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2019-12-22T22:12:46.802 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2019-12-22T22:12:46.900 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2019-12-22T22:12:47.121 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2019-12-22T22:12:47.196 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user 2019-12-22T22:12:47.197 INFO:teuthology.task.ansible.out:": "gabrioux", "validate_certs": true} 2019-12-22T22:12:47.291 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2019-12-22T22:12:47.487 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2019-12-22T22:12:47.550 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2019-12-22T22:12:47.651 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2019-12-22T22:12:47.819 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2019-12-22T22:12:47.884 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2019-12-22T22:12:48.000 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2019-12-22T22:12:48.162 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user 2019-12-22T22:12:48.172 INFO:teuthology.task.ansible.out:": "gabrioux", "validate_certs": true} 2019-12-22T22:12:48.255 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS 2019-12-22T22:12:48.276 INFO:teuthology.task.ansible.out:33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2019-12-22T22:12:48.376 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2019-12-22T22:12:48.378 INFO:teuthology.task.ansible.out: 2019-12-22T22:12:48.489 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2019-12-22T22:12:48.623 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2019-12-22T22:12:48.749 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2019-12-22T22:12:48.856 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2019-12-22T22:12:49.000 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2019-12-22T22:12:49.150 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user 2019-12-22T22:12:49.153 INFO:teuthology.task.ansible.out:": "gabrioux", "validate_certs": true} 2019-12-22T22:12:49.307 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS 2019-12-22T22:12:49.309 INFO:teuthology.task.ansible.out:33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2019-12-22T22:12:49.377 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2019-12-22T22:12:49.527 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2019-12-22T22:12:49.673 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2019-12-22T22:12:49.685 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-22T22:12:49.889 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2019-12-22T22:12:50.028 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2019-12-22T22:12:50.046 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2019-12-22T22:12:50.272 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS 2019-12-22T22:12:50.273 INFO:teuthology.task.ansible.out:33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2019-12-22T22:12:50.360 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2019-12-22T22:12:50.379 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2019-12-22T22:12:50.642 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2019-12-22T22:12:50.696 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2019-12-22T22:12:50.739 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-22T22:12:51.000 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2019-12-22T22:12:51.021 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2019-12-22T22:12:51.106 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2019-12-22T22:12:51.350 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2019-12-22T22:12:51.362 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2019-12-22T22:12:51.475 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2019-12-22T22:12:51.736 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2019-12-22T22:12:51.762 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-22T22:12:51.858 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2019-12-22T22:12:52.085 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2019-12-22T22:12:52.118 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2019-12-22T22:12:52.234 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2019-12-22T22:12:52.426 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2019-12-22T22:12:52.473 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2019-12-22T22:12:52.591 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2019-12-22T22:12:52.805 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2019-12-22T22:12:52.881 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2019-12-22T22:12:52.975 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2019-12-22T22:12:53.169 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2019-12-22T22:12:53.223 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2019-12-22T22:12:53.352 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2019-12-22T22:12:53.545 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2019-12-22T22:12:53.568 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2019-12-22T22:12:53.720 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2019-12-22T22:12:53.938 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2019-12-22T22:12:53.947 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2019-12-22T22:12:54.106 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2019-12-22T22:12:54.285 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2019-12-22T22:12:54.371 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2019-12-22T22:12:54.477 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2019-12-22T22:12:54.647 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2019-12-22T22:12:54.703 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2019-12-22T22:12:54.834 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2019-12-22T22:12:55.005 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2019-12-22T22:12:55.056 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2019-12-22T22:12:55.209 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2019-12-22T22:12:55.359 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2019-12-22T22:12:55.417 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2019-12-22T22:12:55.568 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2019-12-22T22:12:55.666 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2019-12-22T22:12:55.776 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2019-12-22T22:12:55.884 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2019-12-22T22:12:55.993 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2019-12-22T22:12:56.132 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2019-12-22T22:12:56.236 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2019-12-22T22:12:56.326 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2019-12-22T22:12:56.488 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2019-12-22T22:12:56.604 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2019-12-22T22:12:56.689 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2019-12-22T22:12:56.869 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2019-12-22T22:12:56.958 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2019-12-22T22:12:57.011 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2019-12-22T22:12:57.245 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2019-12-22T22:12:57.320 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2019-12-22T22:12:57.388 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.lo 2019-12-22T22:12:57.390 INFO:teuthology.task.ansible.out:caldomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2019-12-22T22:12:57.592 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2019-12-22T22:12:57.672 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfb 2019-12-22T22:12:57.674 INFO:teuthology.task.ansible.out:smXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2019-12-22T22:12:57.737 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2019-12-22T22:12:57.944 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2019-12-22T22:12:58.040 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz1UnJy64hXJKl1OsqRPlZ84m/fyHE8/Hqimy5s8sWcvRs2tL5oK30zFataqv1BC7r46+9t1eGUIRAuBg6QPuJnJCnPzS8yTVJ6DqSDGE2NAnpcMK5fSb+qgQ2adHFkjzZ4eKZbnDJMjYBtpUEp1Ve2ncdq5YCy6hGLIstkGq331ULBc/Lm3ZHBwy5I51S8eldkuP975otBYSPZaaN25QDncZzrW36zb3VwoGUAANEfHC1kLgTTFjx0bdsAGYCduXzyyXls8qSv9gbzcRzZZoH5B7tj8VDBoHszPnqGjGqwtWaelifYE/Svp+8fJKLlzRUM+XGNZoNcRl1/1XVhYV/ deepika@deepika", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2019-12-22T22:12:58.079 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2019-12-22T22:12:58.281 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2019-12-22T22:12:58.419 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2019-12-22T22:12:58.432 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2019-12-22T22:12:58.671 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2019-12-22T22:12:58.756 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2019-12-22T22:12:58.789 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.lo 2019-12-22T22:12:58.793 INFO:teuthology.task.ansible.out:caldomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2019-12-22T22:12:59.023 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2019-12-22T22:12:59.120 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2019-12-22T22:12:59.165 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfb 2019-12-22T22:12:59.166 INFO:teuthology.task.ansible.out:smXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2019-12-22T22:12:59.400 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2019-12-22T22:12:59.477 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2019-12-22T22:12:59.547 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz1UnJy64hXJKl1OsqRPlZ84m/fyHE8/Hqimy5s8sWcvRs2tL5oK30zFataqv1BC7r46+9t1eGUIRAuBg6QPuJnJCnPzS8yTVJ6DqSDGE2NAnpcMK5fSb+qgQ2adHFkjzZ4eKZbnDJMjYBtpUEp1Ve2ncdq5YCy6hGLIstkGq331ULBc/Lm3ZHBwy5I51S8eldkuP975otBYSPZaaN25QDncZzrW36zb3VwoGUAANEfHC1kLgTTFjx0bdsAGYCduXzyyXls8qSv9gbzcRzZZoH5B7tj8VDBoHszPnqGjGqwtWaelifYE/Svp+8fJKLlzRUM+XGNZoNcRl1/1XVhYV/ deepika@deepika", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2019-12-22T22:12:59.755 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.lo 2019-12-22T22:12:59.757 INFO:teuthology.task.ansible.out:caldomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2019-12-22T22:12:59.822 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2019-12-22T22:12:59.911 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2019-12-22T22:13:00.123 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2019-12-22T22:13:00.145 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfb 2019-12-22T22:13:00.147 INFO:teuthology.task.ansible.out:smXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2019-12-22T22:13:00.307 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2019-12-22T22:13:00.466 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2019-12-22T22:13:00.500 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz1UnJy64hXJKl1OsqRPlZ84m/fyHE8/Hqimy5s8sWcvRs2tL5oK30zFataqv1BC7r46+9t1eGUIRAuBg6QPuJnJCnPzS8yTVJ6DqSDGE2NAnpcMK5fSb+qgQ2adHFkjzZ4eKZbnDJMjYBtpUEp1Ve2ncdq5YCy6hGLIstkGq331ULBc/Lm3ZHBwy5I51S8eldkuP975otBYSPZaaN25QDncZzrW36zb3VwoGUAANEfHC1kLgTTFjx0bdsAGYCduXzyyXls8qSv9gbzcRzZZoH5B7tj8VDBoHszPnqGjGqwtWaelifYE/Svp+8fJKLlzRUM+XGNZoNcRl1/1XVhYV/ deepika@deepika", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2019-12-22T22:13:00.659 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2019-12-22T22:13:00.809 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2019-12-22T22:13:00.856 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2019-12-22T22:13:01.002 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2019-12-22T22:13:01.167 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2019-12-22T22:13:01.235 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2019-12-22T22:13:01.368 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2019-12-22T22:13:01.508 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2019-12-22T22:13:01.586 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2019-12-22T22:13:01.741 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2019-12-22T22:13:01.838 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2019-12-22T22:13:01.967 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2019-12-22T22:13:02.117 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2019-12-22T22:13:02.194 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2019-12-22T22:13:02.315 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2019-12-22T22:13:02.491 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2019-12-22T22:13:02.527 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2019-12-22T22:13:02.675 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2019-12-22T22:13:02.849 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2019-12-22T22:13:02.877 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_key 2019-12-22T22:13:02.881 INFO:teuthology.task.ansible.out:s", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-22T22:13:03.036 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2019-12-22T22:13:03.186 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2019-12-22T22:13:03.209 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2019-12-22T22:13:03.404 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2019-12-22T22:13:03.532 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2019-12-22T22:13:03.599 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2019-12-22T22:13:03.773 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2019-12-22T22:13:03.900 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2019-12-22T22:13:03.941 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f', u'name': u'toabctl'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2019-12-22T22:13:04.142 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2019-12-22T22:13:04.253 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2019-12-22T22:13:04.288 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72', u'name': u'mgfritch'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2019-12-22T22:13:04.453 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:04 +0000 (0:00:35.074) 0:01:27.630 ******* 2019-12-22T22:13:04.519 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:04.528 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:04.540 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:04.563 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2019-12-22T22:13:04.564 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:04.566 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:04.570 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:04.581 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:04.625 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-22T22:13:04.627 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_key 2019-12-22T22:13:04.629 INFO:teuthology.task.ansible.out:s", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-22T22:13:04.740 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:04.920 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-22T22:13:04.925 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2019-12-22T22:13:04.976 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2019-12-22T22:13:05.082 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-22T22:13:05.083 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2019-12-22T22:13:05.093 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:05.272 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2019-12-22T22:13:05.356 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2019-12-22T22:13:05.436 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-22T22:13:05.440 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2019-12-22T22:13:05.628 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-22T22:13:05.630 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_key 2019-12-22T22:13:05.632 INFO:teuthology.task.ansible.out:s", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-22T22:13:05.704 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f', u'name': u'toabctl'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2019-12-22T22:13:05.753 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-22T22:13:05.754 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2019-12-22T22:13:06.004 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-22T22:13:06.005 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2019-12-22T22:13:06.059 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72', u'name': u'mgfritch'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2019-12-22T22:13:06.161 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-22T22:13:06.162 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2019-12-22T22:13:06.163 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.215 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:06 +0000 (0:00:01.762) 0:01:29.393 ******* 2019-12-22T22:13:06.228 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-lap 2019-12-22T22:13:06.230 INFO:teuthology.task.ansible.out:top-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.243 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.245 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.268 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.280 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.291 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.303 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.314 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.326 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.338 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.350 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** changed: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2019-12-22T22:13:06.362 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.374 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.386 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.397 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.410 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.421 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.425 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@d 2019-12-22T22:13:06.426 INFO:teuthology.task.ansible.out:esktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.436 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 2019-12-22T22:13:06.464 INFO:teuthology.task.ansible.out: 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.470 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.471 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.482 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.497 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.498 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.509 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.522 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.533 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.544 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.557 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.558 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.571 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.721 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.724 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.726 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.728 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.728 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.730 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.731 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.731 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.732 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.733 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-22T22:13:06.733 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f', u'name': u'toabctl'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5E 2019-12-22T22:13:06.735 INFO:teuthology.task.ansible.out:l54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.748 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.758 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.779 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.790 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.791 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.792 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+r 2019-12-22T22:13:06.793 INFO:teuthology.task.ansible.out:R1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2019-12-22T22:13:06.804 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.816 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.817 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.818 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.828 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.840 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.851 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.864 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.889 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.898 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.916 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.921 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.933 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.944 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.946 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.957 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.968 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.980 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.981 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:06.996 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.005 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.016 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.029 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.040 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.054 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.066 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.067 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.081 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.092 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.104 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** changed: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72', u'name': u'mgfritch'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2019-12-22T22:13:07.120 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.153 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-22T22:13:07.204 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f', u'name': u'toabctl'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.208 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'ovpn': u'mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72', u'name': u'mgfritch'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.232 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2019-12-22T22:13:07.274 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:07 +0000 (0:00:01.058) 0:01:30.452 ******* 2019-12-22T22:13:07.309 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:07 +0000 (0:00:00.034) 0:01:30.486 ******* 2019-12-22T22:13:07.331 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.369 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.398 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:07 +0000 (0:00:00.089) 0:01:30.576 ******* 2019-12-22T22:13:07.407 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.412 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.413 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.414 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.415 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.544 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2019-12-22T22:13:07.765 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2019-12-22T22:13:07.852 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** ok: [smithi091.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2019-12-22T22:13:07.909 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-22T22:13:07.911 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2019-12-22T22:13:07.922 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.934 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.956 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.967 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.979 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:07.990 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.001 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.033 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.035 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.048 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.050 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.062 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.073 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.085 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.105 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.111 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.131 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** ok: [smithi091.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2019-12-22T22:13:08.136 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-22T22:13:08.138 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.148 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2019-12-22T22:13:08.150 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.154 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.176 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.188 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.215 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.216 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.228 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.230 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.249 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.260 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.273 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.287 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.296 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.308 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.352 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.362 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.384 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.408 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.410 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.411 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.427 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.438 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.448 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.458 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.459 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** 2019-12-22T22:13:08.460 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2019-12-22T22:13:08.471 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-22T22:13:08.473 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.483 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.485 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.509 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.525 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.541 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB 2019-12-22T22:13:08.543 INFO:teuthology.task.ansible.out:4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.557 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.569 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.581 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.592 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.609 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.616 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.629 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.641 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.654 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.666 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.702 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.704 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.715 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.729 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.740 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.752 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.770 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.779 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.788 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.792 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.802 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.806 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi091.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2019-12-22T22:13:08.814 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-22T22:13:08.816 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.829 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.830 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.842 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.855 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.856 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.867 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.880 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.893 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.894 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2019-12-22T22:13:08.905 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.917 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.919 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f', u'name': u'toabctl'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.930 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.932 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'ovpn': u'mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72', u'name': u'mgfritch'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:08.951 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.000 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:08 +0000 (0:00:01.601) 0:01:32.178 ******* 2019-12-22T22:13:09.008 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.009 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.010 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.022 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.024 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.034 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.055 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.110 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:09 +0000 (0:00:00.109) 0:01:32.288 ******* 2019-12-22T22:13:09.118 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.120 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.123 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.143 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** ok: [smithi091.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.168 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.184 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.190 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.235 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.236 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.248 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.262 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.273 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.287 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.293 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.301 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.327 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.334 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.346 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.368 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.369 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.371 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.382 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.383 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.401 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.408 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.417 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.421 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.431 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.443 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.443 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.455 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.467 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.478 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.480 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.490 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.502 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.513 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi107.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2019-12-22T22:13:09.514 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.526 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-22T22:13:09.527 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2019-12-22T22:13:09.528 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.550 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.572 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.583 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.594 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.617 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.629 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.652 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.680 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.683 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.701 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.714 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.732 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.744 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.756 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.772 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.786 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.797 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.809 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.820 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.833 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.844 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.856 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.866 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.878 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** ok: [smithi107.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2019-12-22T22:13:09.890 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-22T22:13:09.892 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** 2019-12-22T22:13:09.893 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2019-12-22T22:13:09.904 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-22T22:13:09.907 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.923 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.932 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.941 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.942 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f', u'name': u'toabctl'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:09.986 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'ovpn': u'mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72', u'name': u'mgfritch'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 3KhqWETFJaVsH/I8FOBfUg 0aca6c3c6d45c554bad8ba9853f0bc575c9915eee33ee6be2059568ad09a9d72"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:10.044 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:10 +0000 (0:00:00.922) 0:01:33.210 ******* 2019-12-22T22:13:10.061 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:10.100 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:10 +0000 (0:00:00.068) 0:01:33.278 ******* 2019-12-22T22:13:10.201 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-22T22:13:10.203 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2019-12-22T22:13:10.213 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2019-12-22T22:13:10.495 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2019-12-22T22:13:10.507 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2019-12-22T22:13:10.555 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2019-12-22T22:13:10.804 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2019-12-22T22:13:10.858 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2019-12-22T22:13:10.885 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2019-12-22T22:13:11.124 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2019-12-22T22:13:11.187 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2019-12-22T22:13:11.208 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2019-12-22T22:13:11.513 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2019-12-22T22:13:11.515 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2019-12-22T22:13:11.556 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2019-12-22T22:13:11.824 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2019-12-22T22:13:11.826 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2019-12-22T22:13:11.909 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2019-12-22T22:13:12.130 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2019-12-22T22:13:12.164 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2019-12-22T22:13:12.253 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2019-12-22T22:13:12.440 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2019-12-22T22:13:12.528 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2019-12-22T22:13:12.591 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2019-12-22T22:13:12.772 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2019-12-22T22:13:12.864 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2019-12-22T22:13:12.954 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2019-12-22T22:13:13.113 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2019-12-22T22:13:13.225 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2019-12-22T22:13:13.276 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2019-12-22T22:13:13.449 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2019-12-22T22:13:13.568 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2019-12-22T22:13:13.610 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2019-12-22T22:13:13.782 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2019-12-22T22:13:13.910 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2019-12-22T22:13:13.933 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2019-12-22T22:13:14.070 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2019-12-22T22:13:14.273 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2019-12-22T22:13:14.276 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2019-12-22T22:13:14.403 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2019-12-22T22:13:14.655 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2019-12-22T22:13:14.668 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2019-12-22T22:13:14.767 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2019-12-22T22:13:15.013 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2019-12-22T22:13:15.096 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2019-12-22T22:13:15.305 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jj", "name": "jj", "remove": false, "state": "absent"} 2019-12-22T22:13:15.359 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2019-12-22T22:13:15.415 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2019-12-22T22:13:15.416 INFO:teuthology.task.ansible.out: 2019-12-22T22:13:15.707 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2019-12-22T22:13:15.760 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2019-12-22T22:13:15.855 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "amarangone", "name": "amarangone", "remove": false, "state": "absent"} 2019-12-22T22:13:16.033 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2019-12-22T22:13:16.065 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2019-12-22T22:13:16.359 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2019-12-22T22:13:16.439 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "oprypin", "name": "oprypin", "remove": false, "state": "absent"} 2019-12-22T22:13:16.686 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2019-12-22T22:13:16.718 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jj", "name": "jj", "remove": false, "state": "absent"} 2019-12-22T22:13:16.993 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "adamyanova", "name": "adamyanova", "remove": false, "state": "absent"} 2019-12-22T22:13:17.004 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2019-12-22T22:13:17.276 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "amarangone", "name": "amarangone", "remove": false, "state": "absent"} 2019-12-22T22:13:17.331 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2019-12-22T22:13:17.570 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "sbillah", "name": "sbillah", "remove": false, "state": "absent"} 2019-12-22T22:13:17.874 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "oprypin", "name": "oprypin", "remove": false, "state": "absent"} 2019-12-22T22:13:18.056 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jj", "name": "jj", "remove": false, "state": "absent"} 2019-12-22T22:13:18.143 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "onyb", "name": "onyb", "remove": false, "state": "absent"} 2019-12-22T22:13:18.451 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "adamyanova", "name": "adamyanova", "remove": false, "state": "absent"} 2019-12-22T22:13:18.660 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "amarangone", "name": "amarangone", "remove": false, "state": "absent"} 2019-12-22T22:13:18.714 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jwilliamson", "name": "jwilliamson", "remove": false, "state": "absent"} 2019-12-22T22:13:19.019 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "sbillah", "name": "sbillah", "remove": false, "state": "absent"} 2019-12-22T22:13:19.220 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "oprypin", "name": "oprypin", "remove": false, "state": "absent"} 2019-12-22T22:13:19.284 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "kmroz", "name": "kmroz", "remove": false, "state": "absent"} 2019-12-22T22:13:19.571 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "onyb", "name": "onyb", "remove": false, "state": "absent"} 2019-12-22T22:13:19.797 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "adamyanova", "name": "adamyanova", "remove": false, "state": "absent"} 2019-12-22T22:13:19.887 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "shehbazj", "name": "shehbazj", "remove": false, "state": "absent"} 2019-12-22T22:13:20.196 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jwilliamson", "name": "jwilliamson", "remove": false, "state": "absent"} 2019-12-22T22:13:20.419 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "sbillah", "name": "sbillah", "remove": false, "state": "absent"} 2019-12-22T22:13:20.464 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "remove": false, "state": "absent"} 2019-12-22T22:13:20.781 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "kmroz", "name": "kmroz", "remove": false, "state": "absent"} 2019-12-22T22:13:20.996 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "onyb", "name": "onyb", "remove": false, "state": "absent"} 2019-12-22T22:13:21.061 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "asheplyakov", "name": "asheplyakov", "remove": false, "state": "absent"} 2019-12-22T22:13:21.365 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "shehbazj", "name": "shehbazj", "remove": false, "state": "absent"} 2019-12-22T22:13:21.543 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jwilliamson", "name": "jwilliamson", "remove": false, "state": "absent"} 2019-12-22T22:13:21.669 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "liupan", "name": "liupan", "remove": false, "state": "absent"} 2019-12-22T22:13:21.990 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "remove": false, "state": "absent"} 2019-12-22T22:13:22.001 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2019-12-22T22:13:22.144 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "kmroz", "name": "kmroz", "remove": false, "state": "absent"} 2019-12-22T22:13:22.360 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2019-12-22T22:13:22.586 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "asheplyakov", "name": "asheplyakov", "remove": false, "state": "absent"} 2019-12-22T22:13:22.671 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2019-12-22T22:13:22.750 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "shehbazj", "name": "shehbazj", "remove": false, "state": "absent"} 2019-12-22T22:13:23.001 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2019-12-22T22:13:23.178 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "liupan", "name": "liupan", "remove": false, "state": "absent"} 2019-12-22T22:13:23.335 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2019-12-22T22:13:23.337 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "remove": false, "state": "absent"} 2019-12-22T22:13:23.520 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2019-12-22T22:13:23.627 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2019-12-22T22:13:23.874 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2019-12-22T22:13:23.928 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "asheplyakov", "name": "asheplyakov", "remove": false, "state": "absent"} 2019-12-22T22:13:23.975 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2019-12-22T22:13:24.196 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2019-12-22T22:13:24.507 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2019-12-22T22:13:24.533 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "liupan", "name": "liupan", "remove": false, "state": "absent"} 2019-12-22T22:13:24.592 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "fche", "name": "fche", "remove": false, "state": "absent"} 2019-12-22T22:13:24.670 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:24 +0000 (0:00:14.569) 0:01:47.847 ******* 2019-12-22T22:13:24.818 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2019-12-22T22:13:24.931 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2019-12-22T22:13:25.115 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2019-12-22T22:13:25.276 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2019-12-22T22:13:25.424 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2019-12-22T22:13:25.621 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2019-12-22T22:13:25.776 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "checksum": "bcb7fb0987df16cc4ec0eb20567ca03f52e167d8", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "66384b595a953a298cfd60e4301d14e6", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1577052804.71-29315140151212/source", "state": "file", "uid": 0} 2019-12-22T22:13:25.818 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:25 +0000 (0:00:01.142) 0:01:48.990 ******* 2019-12-22T22:13:25.849 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2019-12-22T22:13:25.851 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => { "msg": "Host smithi091.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2019-12-22T22:13:25.890 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:25 +0000 (0:00:00.078) 0:01:49.068 ******* 2019-12-22T22:13:25.930 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** ok: [smithi091.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2019-12-22T22:13:25.967 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:25 +0000 (0:00:00.076) 0:01:49.145 ******* 2019-12-22T22:13:25.973 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-22T22:13:25.974 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2019-12-22T22:13:25.997 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2019-12-22T22:13:25.999 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:26.059 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:26 +0000 (0:00:00.091) 0:01:49.236 ******* 2019-12-22T22:13:26.065 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-22T22:13:26.065 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "fche", "name": "fche", "remove": false, "state": "absent"} 2019-12-22T22:13:26.136 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:26 +0000 (0:00:00.077) 0:01:49.314 ******* 2019-12-22T22:13:26.268 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2019-12-22T22:13:26.509 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.115153", "end": "2019-12-22 22:13:26.476690", "rc": 0, "start": "2019-12-22 22:13:26.361537", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2019-12-22T22:13:26.579 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:26 +0000 (0:00:00.442) 0:01:49.757 ******* 2019-12-22T22:13:26.616 INFO:teuthology.task.ansible.out: 2019-12-22T22:13:26.618 INFO:teuthology.task.ansible.out:TASK [users : Remove revoked users] ******************************************** 2019-12-22T22:13:26.619 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2019-12-22T22:13:26.622 INFO:teuthology.task.ansible.out: 2019-12-22T22:13:26.623 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:26.679 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:26 +0000 (0:00:00.099) 0:01:49.857 ******* 2019-12-22T22:13:26.727 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2019-12-22T22:13:26.729 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:26.772 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:26 +0000 (0:00:00.089) 0:01:49.946 ******* 2019-12-22T22:13:26.807 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2019-12-22T22:13:26.809 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:26.854 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:26 +0000 (0:00:00.085) 0:01:50.032 ******* 2019-12-22T22:13:26.893 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2019-12-22T22:13:26.894 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:26.945 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:26 +0000 (0:00:00.090) 0:01:50.123 ******* 2019-12-22T22:13:26.959 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi043.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2019-12-22T22:13:26.995 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2019-12-22T22:13:26.997 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:27.033 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:27 +0000 (0:00:00.088) 0:01:50.211 ******* 2019-12-22T22:13:27.066 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:27.103 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:27 +0000 (0:00:00.070) 0:01:50.281 ******* 2019-12-22T22:13:27.152 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2019-12-22T22:13:27.154 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi091.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-22T22:13:27.191 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:27 +0000 (0:00:00.087) 0:01:50.368 ******* 2019-12-22T22:13:27.197 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2019-12-22T22:13:27.197 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "checksum": "bcb7fb0987df16cc4ec0eb20567ca03f52e167d8", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "66384b595a953a298cfd60e4301d14e6", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1577052806.17-2446463305003/source", "state": "file", "uid": 0} 2019-12-22T22:13:27.243 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:27 +0000 (0:00:00.052) 0:01:50.421 ******* 2019-12-22T22:13:27.248 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2019-12-22T22:13:27.249 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:27.288 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:27 +0000 (0:00:00.045) 0:01:50.466 ******* 2019-12-22T22:13:27.298 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** ok: [smithi107.front.sepia.ceph.com] => { "msg": "Host smithi107.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2019-12-22T22:13:27.336 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:27 +0000 (0:00:00.044) 0:01:50.511 ******* 2019-12-22T22:13:27.345 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:27.384 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:27 +0000 (0:00:00.049) 0:01:50.560 ******* 2019-12-22T22:13:27.394 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** ok: [smithi107.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2019-12-22T22:13:27.432 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:27 +0000 (0:00:00.044) 0:01:50.605 ******* 2019-12-22T22:13:27.438 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi091.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-22T22:13:27.475 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:27 +0000 (0:00:00.047) 0:01:50.653 ******* 2019-12-22T22:13:27.484 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:27.523 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:27 +0000 (0:00:00.047) 0:01:50.701 ******* 2019-12-22T22:13:27.532 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2019-12-22T22:13:27.533 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:27.591 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:27 +0000 (0:00:00.064) 0:01:50.765 ******* 2019-12-22T22:13:27.596 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** changed: [smithi043.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "fche", "name": "fche", "remove": false, "state": "absent"} 2019-12-22T22:13:27.620 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2019-12-22T22:13:27.621 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-22T22:13:27.662 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:27 +0000 (0:00:00.070) 0:01:50.835 ******* 2019-12-22T22:13:27.693 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:27 +0000 (0:00:00.035) 0:01:50.870 ******* 2019-12-22T22:13:27.721 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2019-12-22T22:13:27.723 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:27.771 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:27 +0000 (0:00:00.076) 0:01:50.947 ******* 2019-12-22T22:13:27.807 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2019-12-22T22:13:27.808 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:27.854 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:27 +0000 (0:00:00.084) 0:01:51.032 ******* 2019-12-22T22:13:27.882 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2019-12-22T22:13:27.884 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:27.924 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:27 +0000 (0:00:00.070) 0:01:51.102 ******* 2019-12-22T22:13:27.958 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2019-12-22T22:13:27.963 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:28.004 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:27 +0000 (0:00:00.080) 0:01:51.182 ******* 2019-12-22T22:13:28.013 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2019-12-22T22:13:28.014 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.090997", "end": "2019-12-22 22:13:27.921027", "rc": 0, "start": "2019-12-22 22:13:27.830030", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2019-12-22T22:13:28.054 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.048) 0:01:51.231 ******* 2019-12-22T22:13:28.060 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2019-12-22T22:13:28.061 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-22T22:13:28.102 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.047) 0:01:51.278 ******* 2019-12-22T22:13:28.107 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2019-12-22T22:13:28.107 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:28.147 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.046) 0:01:51.324 ******* 2019-12-22T22:13:28.152 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2019-12-22T22:13:28.154 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:28.190 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.042) 0:01:51.367 ******* 2019-12-22T22:13:28.198 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2019-12-22T22:13:28.200 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:28.236 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.046) 0:01:51.414 ******* 2019-12-22T22:13:28.244 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:28.282 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.045) 0:01:51.459 ******* 2019-12-22T22:13:28.292 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2019-12-22T22:13:28.300 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:28.329 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.044) 0:01:51.504 ******* 2019-12-22T22:13:28.332 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2019-12-22T22:13:28.332 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:28.371 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.045) 0:01:51.549 ******* 2019-12-22T22:13:28.377 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2019-12-22T22:13:28.379 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:28.427 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.055) 0:01:51.605 ******* 2019-12-22T22:13:28.437 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:28.472 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.045) 0:01:51.650 ******* 2019-12-22T22:13:28.479 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2019-12-22T22:13:28.480 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:28.518 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.045) 0:01:51.695 ******* 2019-12-22T22:13:28.523 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2019-12-22T22:13:28.523 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:28.573 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.055) 0:01:51.751 ******* 2019-12-22T22:13:28.579 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2019-12-22T22:13:28.581 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:28.617 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.044) 0:01:51.795 ******* 2019-12-22T22:13:28.623 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2019-12-22T22:13:28.624 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:28.667 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.049) 0:01:51.845 ******* 2019-12-22T22:13:28.673 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2019-12-22T22:13:28.675 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi107.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-22T22:13:28.713 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.046) 0:01:51.891 ******* 2019-12-22T22:13:28.719 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2019-12-22T22:13:28.719 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-22T22:13:28.768 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.047) 0:01:51.939 ******* 2019-12-22T22:13:28.770 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2019-12-22T22:13:28.774 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:28.810 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.048) 0:01:51.988 ******* 2019-12-22T22:13:28.816 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2019-12-22T22:13:28.817 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "checksum": "bcb7fb0987df16cc4ec0eb20567ca03f52e167d8", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "66384b595a953a298cfd60e4301d14e6", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1577052807.68-57636136993581/source", "state": "file", "uid": 0} 2019-12-22T22:13:28.818 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2019-12-22T22:13:28.860 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.050) 0:01:52.038 ******* 2019-12-22T22:13:28.897 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.036) 0:01:52.075 ******* 2019-12-22T22:13:28.903 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2019-12-22T22:13:28.905 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Log the OS name, version and release] *************************** ok: [smithi043.front.sepia.ceph.com] => { "msg": "Host smithi043.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2019-12-22T22:13:28.945 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.048) 0:01:52.123 ******* 2019-12-22T22:13:28.980 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:28 +0000 (0:00:00.034) 0:01:52.158 ******* 2019-12-22T22:13:28.986 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2019-12-22T22:13:28.987 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:29.025 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:29 +0000 (0:00:00.044) 0:01:52.203 ******* 2019-12-22T22:13:29.030 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2019-12-22T22:13:29.032 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} TASK [common : Determine if node is registered with subscription-manager.] ***** 2019-12-22T22:13:29.033 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-22T22:13:29.074 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:29 +0000 (0:00:00.049) 0:01:52.252 ******* 2019-12-22T22:13:29.109 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:29 +0000 (0:00:00.034) 0:01:52.287 ******* 2019-12-22T22:13:29.115 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2019-12-22T22:13:29.117 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:29.154 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:29 +0000 (0:00:00.045) 0:01:52.332 ******* 2019-12-22T22:13:29.159 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2019-12-22T22:13:29.161 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:29.203 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:29 +0000 (0:00:00.048) 0:01:52.380 ******* 2019-12-22T22:13:29.232 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2019-12-22T22:13:29.234 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-22T22:13:29.268 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:29 +0000 (0:00:00.065) 0:01:52.446 ******* 2019-12-22T22:13:29.305 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2019-12-22T22:13:29.307 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:29.349 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:29 +0000 (0:00:00.080) 0:01:52.527 ******* 2019-12-22T22:13:29.385 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2019-12-22T22:13:29.386 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-22T22:13:29.387 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:29.431 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:29 +0000 (0:00:00.082) 0:01:52.609 ******* 2019-12-22T22:13:29.464 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:29 +0000 (0:00:00.033) 0:01:52.642 ******* 2019-12-22T22:13:29.495 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:29.530 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:29 +0000 (0:00:00.065) 0:01:52.707 ******* 2019-12-22T22:13:29.582 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:29.599 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:29 +0000 (0:00:00.065) 0:01:52.773 ******* 2019-12-22T22:13:29.602 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.107298", "end": "2019-12-22 22:13:29.558687", "rc": 0, "start": "2019-12-22 22:13:29.451389", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2019-12-22T22:13:29.640 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:29 +0000 (0:00:00.045) 0:01:52.818 ******* 2019-12-22T22:13:29.648 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi107.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-22T22:13:29.685 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:29 +0000 (0:00:00.044) 0:01:52.863 ******* 2019-12-22T22:13:29.690 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2019-12-22T22:13:29.691 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:29.730 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:29 +0000 (0:00:00.044) 0:01:52.908 ******* 2019-12-22T22:13:29.735 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2019-12-22T22:13:29.745 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:29.776 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:29 +0000 (0:00:00.042) 0:01:52.951 ******* 2019-12-22T22:13:29.784 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:29.815 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:29 +0000 (0:00:00.042) 0:01:52.993 ******* 2019-12-22T22:13:29.827 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:29.860 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:29 +0000 (0:00:00.044) 0:01:53.038 ******* 2019-12-22T22:13:29.866 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2019-12-22T22:13:29.867 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:29.910 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:29 +0000 (0:00:00.050) 0:01:53.088 ******* 2019-12-22T22:13:29.920 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:29.958 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:29 +0000 (0:00:00.047) 0:01:53.136 ******* 2019-12-22T22:13:29.964 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2019-12-22T22:13:29.965 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:30.006 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:30 +0000 (0:00:00.047) 0:01:53.183 ******* 2019-12-22T22:13:30.011 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2019-12-22T22:13:30.013 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:30.050 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:30 +0000 (0:00:00.044) 0:01:53.228 ******* 2019-12-22T22:13:30.058 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2019-12-22T22:13:30.059 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:30.104 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:30 +0000 (0:00:00.053) 0:01:53.282 ******* 2019-12-22T22:13:30.116 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:30.162 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:30 +0000 (0:00:00.058) 0:01:53.340 ******* 2019-12-22T22:13:30.172 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:30.210 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:30 +0000 (0:00:00.047) 0:01:53.388 ******* 2019-12-22T22:13:30.218 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2019-12-22T22:13:30.220 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:30.269 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:30 +0000 (0:00:00.058) 0:01:53.447 ******* 2019-12-22T22:13:30.274 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2019-12-22T22:13:30.274 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-22T22:13:30.277 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-22T22:13:30.319 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:30 +0000 (0:00:00.050) 0:01:53.497 ******* 2019-12-22T22:13:30.325 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2019-12-22T22:13:30.325 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-22T22:13:30.371 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:30 +0000 (0:00:00.051) 0:01:53.549 ******* 2019-12-22T22:13:30.376 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2019-12-22T22:13:30.377 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:30.420 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:30 +0000 (0:00:00.043) 0:01:53.592 ******* 2019-12-22T22:13:30.423 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2019-12-22T22:13:30.463 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:30 +0000 (0:00:00.048) 0:01:53.641 ******* 2019-12-22T22:13:30.468 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2019-12-22T22:13:30.469 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:30.506 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:30 +0000 (0:00:00.042) 0:01:53.683 ******* 2019-12-22T22:13:30.510 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2019-12-22T22:13:30.512 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:30.556 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:30 +0000 (0:00:00.047) 0:01:53.731 ******* 2019-12-22T22:13:30.560 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi043.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-22T22:13:30.600 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:30 +0000 (0:00:00.046) 0:01:53.778 ******* 2019-12-22T22:13:30.659 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2019-12-22T22:13:30.661 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:30.697 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:30 +0000 (0:00:00.097) 0:01:53.875 ******* 2019-12-22T22:13:30.703 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2019-12-22T22:13:30.704 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "1d6dcd385d1b7c9c2b757c62b841c887e90d7f66", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 293, "state": "file", "uid": 0} 2019-12-22T22:13:30.727 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2019-12-22T22:13:30.729 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-22T22:13:30.775 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:30 +0000 (0:00:00.077) 0:01:53.953 ******* 2019-12-22T22:13:30.818 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:30.850 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:30 +0000 (0:00:00.074) 0:01:54.027 ******* 2019-12-22T22:13:30.882 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:30.918 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:30 +0000 (0:00:00.068) 0:01:54.096 ******* 2019-12-22T22:13:30.924 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2019-12-22T22:13:30.925 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-22T22:13:30.965 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:30 +0000 (0:00:00.047) 0:01:54.143 ******* 2019-12-22T22:13:30.972 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2019-12-22T22:13:30.973 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:31.010 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:31 +0000 (0:00:00.044) 0:01:54.188 ******* 2019-12-22T22:13:31.050 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2019-12-22T22:13:31.052 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:31.102 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:31 +0000 (0:00:00.091) 0:01:54.279 ******* 2019-12-22T22:13:31.129 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi043.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-22T22:13:31.173 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:31 +0000 (0:00:00.070) 0:01:54.350 ******* 2019-12-22T22:13:31.208 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2019-12-22T22:13:31.210 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:31.250 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:31 +0000 (0:00:00.077) 0:01:54.427 ******* 2019-12-22T22:13:31.277 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2019-12-22T22:13:31.278 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:31.335 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:31 +0000 (0:00:00.085) 0:01:54.513 ******* 2019-12-22T22:13:31.373 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2019-12-22T22:13:31.375 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:31.410 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:31 +0000 (0:00:00.075) 0:01:54.588 ******* 2019-12-22T22:13:31.461 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:31.488 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:31 +0000 (0:00:00.077) 0:01:54.666 ******* 2019-12-22T22:13:31.518 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2019-12-22T22:13:31.527 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:31.585 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:31 +0000 (0:00:00.096) 0:01:54.762 ******* 2019-12-22T22:13:31.620 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:31.654 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:31 +0000 (0:00:00.069) 0:01:54.832 ******* 2019-12-22T22:13:31.661 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2019-12-22T22:13:31.662 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "d9691278f687ee76cfe8a3de60fcbd4f1f79a8c6", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 327, "state": "file", "uid": 0} 2019-12-22T22:13:31.703 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:31 +0000 (0:00:00.048) 0:01:54.881 ******* 2019-12-22T22:13:31.708 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2019-12-22T22:13:31.709 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-22T22:13:31.746 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:31 +0000 (0:00:00.043) 0:01:54.924 ******* 2019-12-22T22:13:31.776 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2019-12-22T22:13:31.814 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:31 +0000 (0:00:00.067) 0:01:54.992 ******* 2019-12-22T22:13:31.867 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:31.887 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:31 +0000 (0:00:00.073) 0:01:55.065 ******* 2019-12-22T22:13:31.945 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** ok: [smithi107.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "1d6dcd385d1b7c9c2b757c62b841c887e90d7f66", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 293, "state": "file", "uid": 0} 2019-12-22T22:13:32.243 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2019-12-22T22:13:32.245 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-22T22:13:32.283 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:32 +0000 (0:00:00.395) 0:01:55.460 ******* 2019-12-22T22:13:32.789 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2019-12-22T22:13:32.790 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "d9691278f687ee76cfe8a3de60fcbd4f1f79a8c6", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 327, "state": "file", "uid": 0} 2019-12-22T22:13:32.838 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:32 +0000 (0:00:00.550) 0:01:56.011 ******* 2019-12-22T22:13:32.857 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* changed: [smithi091.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "md5sum": "6467980f3cbb5aa63a3cfe239626f53e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1577052811.74-156927629322061/source", "state": "file", "uid": 0} 2019-12-22T22:13:33.282 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2019-12-22T22:13:33.284 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "1d6dcd385d1b7c9c2b757c62b841c887e90d7f66", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 293, "state": "file", "uid": 0} 2019-12-22T22:13:33.980 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* changed: [smithi091.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "md5sum": "0848bb8607c5b0c5706d8e1930fa7d55", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1577052812.85-23806786299460/source", "state": "file", "uid": 0} 2019-12-22T22:13:34.027 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:34 +0000 (0:00:01.189) 0:01:57.201 ******* 2019-12-22T22:13:34.092 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "md5sum": "6467980f3cbb5aa63a3cfe239626f53e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1577052812.89-205307061128226/source", "state": "file", "uid": 0} 2019-12-22T22:13:34.265 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2019-12-22T22:13:34.267 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "d9691278f687ee76cfe8a3de60fcbd4f1f79a8c6", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 327, "state": "file", "uid": 0} 2019-12-22T22:13:34.319 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:34 +0000 (0:00:00.295) 0:01:57.497 ******* 2019-12-22T22:13:34.528 INFO:teuthology.task.ansible.out: TASK [common : Temporarily comment mirrorlist so we can update yum cache] ****** 2019-12-22T22:13:34.529 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made", "warnings": ["Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually"]} 2019-12-22T22:13:35.004 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} [WARNING]: Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually 2019-12-22T22:13:35.041 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:35 +0000 (0:00:00.721) 0:01:58.219 ******* 2019-12-22T22:13:35.257 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2019-12-22T22:13:35.257 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "md5sum": "0848bb8607c5b0c5706d8e1930fa7d55", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1577052814.02-9437377247998/source", "state": "file", "uid": 0} 2019-12-22T22:13:35.298 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:35 +0000 (0:00:00.256) 0:01:58.476 ******* 2019-12-22T22:13:35.492 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "md5sum": "6467980f3cbb5aa63a3cfe239626f53e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1577052814.36-51288112286883/source", "state": "file", "uid": 0} 2019-12-22T22:13:35.654 INFO:teuthology.task.ansible.out: TASK [common : Temporarily comment mirrorlist so we can update yum cache] ****** 2019-12-22T22:13:35.656 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made", "warnings": ["Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually"]} 2019-12-22T22:13:35.966 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-22T22:13:36.011 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:36 +0000 (0:00:00.713) 0:01:59.189 ******* 2019-12-22T22:13:36.652 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* changed: [smithi043.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "md5sum": "0848bb8607c5b0c5706d8e1930fa7d55", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1577052815.52-201047662624438/source", "state": "file", "uid": 0} 2019-12-22T22:13:36.693 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:36 +0000 (0:00:00.678) 0:01:59.868 ******* 2019-12-22T22:13:37.047 INFO:teuthology.task.ansible.out: TASK [common : Temporarily comment mirrorlist so we can update yum cache] ****** changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made", "warnings": ["Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually"]} 2019-12-22T22:13:37.384 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-22T22:13:37.430 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:37 +0000 (0:00:00.732) 0:02:00.601 ******* 2019-12-22T22:13:47.618 INFO:teuthology.task.ansible.out: TASK [common : Update yum cache] *********************************************** [WARNING]: Consider using the file module with state=absent rather than running 'rm'. If you need to use command because file is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. 2019-12-22T22:13:47.619 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": "rm -rf /var/cache/yum/*; yum clean all; yum makecache", "delta": "0:00:12.256077", "end": "2019-12-22 22:13:47.578517", "rc": 0, "start": "2019-12-22 22:13:35.322440", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: mirror.umd.edu\n * extras: mirror.fileplanet.com\n * updates: repos-va.psychz.net\nMetadata Cache Created", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Loaded plugins: fastestmirror, langpacks, priorities", "Determining fastest mirrors", " * base: mirror.umd.edu", " * extras: mirror.fileplanet.com", " * updates: repos-va.psychz.net", "Metadata Cache Created"]} 2019-12-22T22:13:47.669 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:47 +0000 (0:00:10.246) 0:02:10.847 ******* 2019-12-22T22:13:48.002 INFO:teuthology.task.ansible.out: 2019-12-22T22:13:48.003 INFO:teuthology.task.ansible.out:TASK [common : Uncomment mirrorlist] ******************************************* changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made"} 2019-12-22T22:13:48.311 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-22T22:13:48.368 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:48 +0000 (0:00:00.698) 0:02:11.546 ******* 2019-12-22T22:13:48.416 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-22T22:13:48.417 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Update yum cache] *********************************************** changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "cmd": "rm -rf /var/cache/yum/*; yum clean all; yum makecache", "delta": "0:00:12.082169", "end": "2019-12-22 22:13:48.388751", "rc": 0, "start": "2019-12-22 22:13:36.306582", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: mirror.facebook.net\n * extras: mirror.fileplanet.com\n * updates: repos-va.psychz.net\nMetadata Cache Created", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Loaded plugins: fastestmirror, langpacks, priorities", "Determining fastest mirrors", " * base: mirror.facebook.net", " * extras: mirror.fileplanet.com", " * updates: repos-va.psychz.net", "Metadata Cache Created"]} 2019-12-22T22:13:48.455 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:48 +0000 (0:00:00.087) 0:02:11.633 ******* 2019-12-22T22:13:48.489 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:48 +0000 (0:00:00.033) 0:02:11.667 ******* 2019-12-22T22:13:48.494 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2019-12-22T22:13:48.495 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:48.534 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:48 +0000 (0:00:00.044) 0:02:11.711 ******* 2019-12-22T22:13:48.560 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2019-12-22T22:13:48.562 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:48.601 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:48 +0000 (0:00:00.067) 0:02:11.779 ******* 2019-12-22T22:13:48.628 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-22T22:13:48.629 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:48.665 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:48 +0000 (0:00:00.064) 0:02:11.843 ******* 2019-12-22T22:13:48.692 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2019-12-22T22:13:48.694 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:48.731 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:48 +0000 (0:00:00.065) 0:02:11.909 ******* 2019-12-22T22:13:48.758 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2019-12-22T22:13:48.760 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:48.798 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:48 +0000 (0:00:00.067) 0:02:11.976 ******* 2019-12-22T22:13:48.825 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2019-12-22T22:13:48.827 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:48.867 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:48 +0000 (0:00:00.068) 0:02:12.045 ******* 2019-12-22T22:13:48.876 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made"} 2019-12-22T22:13:48.899 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:48.937 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:48 +0000 (0:00:00.070) 0:02:12.115 ******* 2019-12-22T22:13:49.152 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-22T22:13:49.187 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:49 +0000 (0:00:00.250) 0:02:12.365 ******* 2019-12-22T22:13:49.214 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-22T22:13:49.216 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:49.251 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:49 +0000 (0:00:00.063) 0:02:12.428 ******* 2019-12-22T22:13:49.278 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2019-12-22T22:13:49.279 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:49.316 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:49 +0000 (0:00:00.065) 0:02:12.494 ******* 2019-12-22T22:13:49.347 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:49.378 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:49 +0000 (0:00:00.062) 0:02:12.556 ******* 2019-12-22T22:13:49.404 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-22T22:13:49.406 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:49.441 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:49 +0000 (0:00:00.063) 0:02:12.619 ******* 2019-12-22T22:13:49.468 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:49.503 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:49 +0000 (0:00:00.061) 0:02:12.681 ******* 2019-12-22T22:13:49.524 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2019-12-22T22:13:49.535 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:49.561 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:49 +0000 (0:00:00.057) 0:02:12.739 ******* 2019-12-22T22:13:49.567 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2019-12-22T22:13:49.567 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.283740", "end": "2019-12-22 22:13:49.500489", "rc": 0, "start": "2019-12-22 22:13:49.216749", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:13:49.605 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:49 +0000 (0:00:00.043) 0:02:12.783 ******* 2019-12-22T22:13:49.610 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2019-12-22T22:13:49.612 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:49.649 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:49 +0000 (0:00:00.044) 0:02:12.827 ******* 2019-12-22T22:13:49.676 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2019-12-22T22:13:49.678 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:49.712 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:49 +0000 (0:00:00.062) 0:02:12.890 ******* 2019-12-22T22:13:50.449 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2019-12-22T22:13:50.449 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2019-12-22T22:13:50.452 INFO:teuthology.task.ansible.out: TASK [common : Update yum cache] *********************************************** changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "cmd": "rm -rf /var/cache/yum/*; yum clean all; yum makecache", "delta": "0:00:12.700195", "end": "2019-12-22 22:13:50.415257", "rc": 0, "start": "2019-12-22 22:13:37.715062", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: mirror.umd.edu\n * extras: mirror.fileplanet.com\n * updates: repos-va.psychz.net\nMetadata Cache Created", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Loaded plugins: fastestmirror, langpacks, priorities", "Determining fastest mirrors", " * base: mirror.umd.edu", " * extras: mirror.fileplanet.com", " * updates: repos-va.psychz.net", "Metadata Cache Created"]} 2019-12-22T22:13:50.491 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:50 +0000 (0:00:00.778) 0:02:13.669 ******* 2019-12-22T22:13:50.529 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:50 +0000 (0:00:00.037) 0:02:13.707 ******* 2019-12-22T22:13:50.558 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2019-12-22T22:13:50.558 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:50.595 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:50 +0000 (0:00:00.066) 0:02:13.773 ******* 2019-12-22T22:13:50.627 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2019-12-22T22:13:50.629 INFO:teuthology.task.ansible.out: skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:50.678 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:50 +0000 (0:00:00.082) 0:02:13.856 ******* 2019-12-22T22:13:50.705 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2019-12-22T22:13:50.707 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:50.755 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:50 +0000 (0:00:00.077) 0:02:13.933 ******* 2019-12-22T22:13:50.782 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2019-12-22T22:13:50.784 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:50.822 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:50 +0000 (0:00:00.066) 0:02:14.000 ******* 2019-12-22T22:13:50.901 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* 2019-12-22T22:13:50.902 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made"} 2019-12-22T22:13:51.207 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-22T22:13:51.245 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:51 +0000 (0:00:00.423) 0:02:14.423 ******* 2019-12-22T22:13:51.273 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-22T22:13:51.274 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:51.309 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:51 +0000 (0:00:00.064) 0:02:14.487 ******* 2019-12-22T22:13:51.336 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2019-12-22T22:13:51.338 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:51.374 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:51 +0000 (0:00:00.064) 0:02:14.552 ******* 2019-12-22T22:13:51.409 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:51.438 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:51 +0000 (0:00:00.064) 0:02:14.616 ******* 2019-12-22T22:13:51.476 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-22T22:13:51.477 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:51.513 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:51 +0000 (0:00:00.074) 0:02:14.691 ******* 2019-12-22T22:13:51.530 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2019-12-22T22:13:51.530 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:01.507464", "end": "2019-12-22 22:13:51.500444", "rc": 0, "start": "2019-12-22 22:13:49.992980", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:13:51.568 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:51 +0000 (0:00:00.054) 0:02:14.746 ******* 2019-12-22T22:13:51.574 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2019-12-22T22:13:51.574 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:51.612 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:51 +0000 (0:00:00.044) 0:02:14.790 ******* 2019-12-22T22:13:51.650 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:51.690 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:51 +0000 (0:00:00.077) 0:02:14.868 ******* 2019-12-22T22:13:51.718 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:51.720 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2019-12-22T22:13:51.760 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:51 +0000 (0:00:00.069) 0:02:14.937 ******* 2019-12-22T22:13:51.806 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:51 +0000 (0:00:00.040) 0:02:14.978 ******* 2019-12-22T22:13:51.807 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:51.845 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:51 +0000 (0:00:00.045) 0:02:15.023 ******* 2019-12-22T22:13:52.073 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2019-12-22T22:13:52.142 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:52 +0000 (0:00:00.295) 0:02:15.319 ******* 2019-12-22T22:13:52.171 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2019-12-22T22:13:52.172 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:52.208 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:52 +0000 (0:00:00.065) 0:02:15.385 ******* 2019-12-22T22:13:52.234 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2019-12-22T22:13:52.235 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:52.274 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:52 +0000 (0:00:00.065) 0:02:15.451 ******* 2019-12-22T22:13:52.299 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2019-12-22T22:13:52.301 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:52.336 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:52 +0000 (0:00:00.063) 0:02:15.514 ******* 2019-12-22T22:13:52.362 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2019-12-22T22:13:52.364 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:52.399 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:52 +0000 (0:00:00.063) 0:02:15.577 ******* 2019-12-22T22:13:53.270 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2019-12-22T22:13:53.316 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:53 +0000 (0:00:00.912) 0:02:16.489 ******* 2019-12-22T22:13:53.527 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2019-12-22T22:13:53.528 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:01.359175", "end": "2019-12-22 22:13:53.500462", "rc": 0, "start": "2019-12-22 22:13:52.141287", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:13:53.565 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:53 +0000 (0:00:00.253) 0:02:16.743 ******* 2019-12-22T22:13:54.095 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2019-12-22T22:13:54.096 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2019-12-22T22:13:54.147 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:54 +0000 (0:00:00.581) 0:02:17.325 ******* 2019-12-22T22:13:54.188 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2019-12-22T22:13:54.190 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:54.228 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:54 +0000 (0:00:00.081) 0:02:17.406 ******* 2019-12-22T22:13:54.259 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2019-12-22T22:13:54.261 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:54.322 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:54 +0000 (0:00:00.080) 0:02:17.487 ******* 2019-12-22T22:13:54.336 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2019-12-22T22:13:54.337 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:54.373 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:54 +0000 (0:00:00.063) 0:02:17.551 ******* 2019-12-22T22:13:54.399 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2019-12-22T22:13:54.401 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:13:54.438 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:54 +0000 (0:00:00.065) 0:02:17.616 ******* 2019-12-22T22:13:55.350 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2019-12-22T22:13:55.351 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2019-12-22T22:13:55.414 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:55 +0000 (0:00:00.975) 0:02:18.592 ******* 2019-12-22T22:13:55.814 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing smartmontools are up to date", ""]} 2019-12-22T22:13:55.852 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:55 +0000 (0:00:00.437) 0:02:19.029 ******* 2019-12-22T22:13:56.826 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2019-12-22T22:13:56.828 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2019-12-22T22:13:57.325 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2019-12-22T22:13:57.326 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing smartmontools are up to date", ""]} 2019-12-22T22:13:57.367 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:57 +0000 (0:00:01.516) 0:02:20.545 ******* 2019-12-22T22:13:58.351 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2019-12-22T22:13:58.353 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2019-12-22T22:13:59.022 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2019-12-22T22:13:59.261 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2019-12-22T22:13:59.451 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing smartmontools are up to date", ""]} 2019-12-22T22:13:59.491 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:59 +0000 (0:00:02.122) 0:02:22.668 ******* 2019-12-22T22:13:59.930 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2019-12-22T22:13:59.930 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2019-12-22T22:13:59.976 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:13:59 +0000 (0:00:00.480) 0:02:23.148 ******* 2019-12-22T22:14:00.226 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2019-12-22T22:14:00.292 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:00 +0000 (0:00:00.321) 0:02:23.469 ******* 2019-12-22T22:14:00.302 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2019-12-22T22:14:00.303 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-22T22:14:00.348 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:00 +0000 (0:00:00.057) 0:02:23.526 ******* 2019-12-22T22:14:00.547 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2019-12-22T22:14:00.547 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2019-12-22T22:14:00.632 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2019-12-22T22:14:00.633 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-22T22:14:00.677 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:00 +0000 (0:00:00.328) 0:02:23.855 ******* 2019-12-22T22:14:01.227 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-12-22T22:14:01.228 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2019-12-22T22:14:01.523 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2019-12-22T22:14:01.525 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2019-12-22T22:14:01.601 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-12-22T22:14:01.602 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2019-12-22T22:14:02.107 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2019-12-22T22:14:02.441 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi043.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2019-12-22T22:14:02.473 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:02 +0000 (0:00:01.795) 0:02:25.651 ******* 2019-12-22T22:14:02.480 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-12-22T22:14:02.481 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2019-12-22T22:14:02.811 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-22T22:14:02.856 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:02 +0000 (0:00:00.380) 0:02:26.031 ******* 2019-12-22T22:14:02.960 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi091.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2019-12-22T22:14:03.014 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:03 +0000 (0:00:00.159) 0:02:26.191 ******* 2019-12-22T22:14:03.059 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:14:03.096 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:03 +0000 (0:00:00.083) 0:02:26.274 ******* 2019-12-22T22:14:03.140 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2019-12-22T22:14:03.174 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:03 +0000 (0:00:00.077) 0:02:26.351 ******* 2019-12-22T22:14:03.380 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi107.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2019-12-22T22:14:03.425 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:03 +0000 (0:00:00.251) 0:02:26.603 ******* 2019-12-22T22:14:03.454 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2019-12-22T22:14:03.456 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:14:03.492 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:03 +0000 (0:00:00.067) 0:02:26.670 ******* 2019-12-22T22:14:03.561 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2019-12-22T22:14:03.562 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2019-12-22T22:14:03.599 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:03 +0000 (0:00:00.106) 0:02:26.777 ******* 2019-12-22T22:14:03.817 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-12-22T22:14:03.818 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2019-12-22T22:14:03.934 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* changed: [smithi107.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2019-12-22T22:14:03.980 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:03 +0000 (0:00:00.381) 0:02:27.158 ******* 2019-12-22T22:14:04.038 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2019-12-22T22:14:04.040 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-22T22:14:04.053 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-22T22:14:04.066 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-22T22:14:04.075 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-22T22:14:04.117 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:04 +0000 (0:00:00.137) 0:02:27.295 ******* 2019-12-22T22:14:04.477 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2019-12-22T22:14:04.478 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2019-12-22T22:14:04.518 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:04 +0000 (0:00:00.401) 0:02:27.696 ******* 2019-12-22T22:14:04.559 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2019-12-22T22:14:04.560 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-22T22:14:04.586 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-22T22:14:04.593 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-22T22:14:04.616 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-22T22:14:04.660 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:04 +0000 (0:00:00.135) 0:02:27.832 ******* 2019-12-22T22:14:04.756 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi043.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2019-12-22T22:14:05.711 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2019-12-22T22:14:05.765 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:05 +0000 (0:00:01.110) 0:02:28.943 ******* 2019-12-22T22:14:05.808 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:14:05.848 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:05 +0000 (0:00:00.083) 0:02:29.026 ******* 2019-12-22T22:14:05.898 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** ok: [smithi043.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2019-12-22T22:14:05.933 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:05 +0000 (0:00:00.084) 0:02:29.110 ******* 2019-12-22T22:14:06.300 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2019-12-22T22:14:06.301 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2019-12-22T22:14:06.349 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:06 +0000 (0:00:00.416) 0:02:29.527 ******* 2019-12-22T22:14:06.405 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** skipping: [smithi043.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-22T22:14:06.417 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-22T22:14:06.427 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-22T22:14:06.449 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-22T22:14:06.489 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:06 +0000 (0:00:00.140) 0:02:29.667 ******* 2019-12-22T22:14:07.439 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2019-12-22T22:14:07.441 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing nagios-common are up to date", "All packages providing nrpe are up to date", "All packages providing nagios-plugins are up to date", "All packages providing nagios-plugins-load are up to date", ""]} 2019-12-22T22:14:07.488 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:07 +0000 (0:00:00.998) 0:02:30.666 ******* 2019-12-22T22:14:07.515 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu)] ***************** 2019-12-22T22:14:07.517 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:14:07.560 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:07 +0000 (0:00:00.071) 0:02:30.738 ******* 2019-12-22T22:14:08.063 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing nagios-common are up to date", "All packages providing nrpe are up to date", "All packages providing nagios-plugins are up to date", "All packages providing nagios-plugins-load are up to date", ""]} 2019-12-22T22:14:08.103 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:08 +0000 (0:00:00.542) 0:02:31.280 ******* 2019-12-22T22:14:08.149 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu)] ***************** 2019-12-22T22:14:08.150 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:14:08.193 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:08 +0000 (0:00:00.090) 0:02:31.371 ******* 2019-12-22T22:14:08.535 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2019-12-22T22:14:08.536 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2019-12-22T22:14:08.581 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:08 +0000 (0:00:00.387) 0:02:31.759 ******* 2019-12-22T22:14:09.050 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2019-12-22T22:14:09.089 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:09 +0000 (0:00:00.508) 0:02:32.267 ******* 2019-12-22T22:14:09.509 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2019-12-22T22:14:09.511 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2019-12-22T22:14:09.572 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:09 +0000 (0:00:00.479) 0:02:32.747 ******* 2019-12-22T22:14:09.612 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:14:09.649 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:09 +0000 (0:00:00.079) 0:02:32.827 ******* 2019-12-22T22:14:09.977 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* ok: [smithi107.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-22T22:14:10.032 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:10 +0000 (0:00:00.382) 0:02:33.210 ******* 2019-12-22T22:14:10.038 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2019-12-22T22:14:10.039 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing nagios-common are up to date", "All packages providing nrpe are up to date", "All packages providing nagios-plugins are up to date", "All packages providing nagios-plugins-load are up to date", ""]} 2019-12-22T22:14:10.116 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:10 +0000 (0:00:00.083) 0:02:33.294 ******* 2019-12-22T22:14:10.124 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2019-12-22T22:14:10.126 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2019-12-22T22:14:10.164 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:10 +0000 (0:00:00.047) 0:02:33.341 ******* 2019-12-22T22:14:10.171 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu)] ***************** 2019-12-22T22:14:10.173 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:14:10.211 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:10 +0000 (0:00:00.046) 0:02:33.388 ******* 2019-12-22T22:14:10.218 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:14:10.256 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:10 +0000 (0:00:00.045) 0:02:33.434 ******* 2019-12-22T22:14:10.430 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2019-12-22T22:14:10.431 INFO:teuthology.task.ansible.out:fatal: [smithi107.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2019-12-22T22:14:10.468 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:10 +0000 (0:00:00.211) 0:02:33.646 ******* 2019-12-22T22:14:10.494 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2019-12-22T22:14:10.496 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:14:10.546 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:10 +0000 (0:00:00.078) 0:02:33.724 ******* 2019-12-22T22:14:10.584 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2019-12-22T22:14:10.585 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-22T22:14:10.623 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:10 +0000 (0:00:00.076) 0:02:33.801 ******* 2019-12-22T22:14:11.010 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2019-12-22T22:14:11.012 INFO:teuthology.task.ansible.out:fatal: [smithi091.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2019-12-22T22:14:11.049 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:11 +0000 (0:00:00.426) 0:02:34.227 ******* 2019-12-22T22:14:11.091 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2019-12-22T22:14:11.093 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:14:11.130 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:11 +0000 (0:00:00.081) 0:02:34.308 ******* 2019-12-22T22:14:11.152 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2019-12-22T22:14:11.192 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:11 +0000 (0:00:00.056) 0:02:34.365 ******* 2019-12-22T22:14:11.484 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2019-12-22T22:14:11.485 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2019-12-22T22:14:11.543 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:11 +0000 (0:00:00.355) 0:02:34.721 ******* 2019-12-22T22:14:12.104 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2019-12-22T22:14:12.155 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:12 +0000 (0:00:00.612) 0:02:35.333 ******* 2019-12-22T22:14:12.172 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2019-12-22T22:14:12.203 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:12 +0000 (0:00:00.048) 0:02:35.381 ******* 2019-12-22T22:14:12.241 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2019-12-22T22:14:12.243 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:14:12.279 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:12 +0000 (0:00:00.076) 0:02:35.457 ******* 2019-12-22T22:14:12.516 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2019-12-22T22:14:12.519 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2019-12-22 22:11:22 UTC", "ActiveEnterTimestampMonotonic": "21158277", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "time-sync.target basic.target network.target systemd-journald.socket nss-lookup.target var-run.mount local-fs.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sun 2019-12-22 22:11:22 UTC", "AssertTimestampMonotonic": "21157529", "Before": "xdm.service shutdown.target multi-user.target getty@tty1.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2019-12-22 22:11:22 UTC", "ConditionTimestampMonotonic": "21157528", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8934", "ExecMainStartTimestamp": "Sun 2019-12-22 22:11:22 UTC", "ExecMainStartTimestampMonotonic": "21158228", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_err 2019-12-22T22:14:12.521 INFO:teuthology.task.ansible.out:ors=no ; start_time=[Sun 2019-12-22 22:11:22 UTC] ; stop_time=[n/a] ; pid=8934 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2019-12-22 22:11:22 UTC", "InactiveExitTimestampMonotonic": "21158277", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127510", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127510", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "8934", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "network-online.target basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOn 2019-12-22T22:14:12.525 INFO:teuthology.task.ansible.out:ly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Sun 2019-12-22 22:11:22 UTC", "WatchdogTimestampMonotonic": "21158260", "WatchdogUSec": "0"}} 2019-12-22T22:14:12.560 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:12 +0000 (0:00:00.280) 0:02:35.737 ******* 2019-12-22T22:14:12.626 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2019-12-22T22:14:12.627 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-22T22:14:12.665 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:12 +0000 (0:00:00.104) 0:02:35.842 ******* 2019-12-22T22:14:12.673 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2019-12-22T22:14:12.675 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2019-12-22 22:11:07 UTC", "ActiveEnterTimestampMonotonic": "19300701", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target var-run.mount basic.target local-fs.target nss-lookup.target systemd-journald.socket time-sync.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sun 2019-12-22 22:11:07 UTC", "AssertTimestampMonotonic": "19299919", "Before": "shutdown.target multi-user.target getty@tty1.service xdm.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2019-12-22 22:11:07 UTC", "ConditionTimestampMonotonic": "19299919", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8964", "ExecMainStartTimestamp": "Sun 2019-12-22 22:11:07 UTC", "ExecMainStartTimestampMonotonic": "19300631", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_err 2019-12-22T22:14:12.677 INFO:teuthology.task.ansible.out:ors=no ; start_time=[Sun 2019-12-22 22:11:07 UTC] ; stop_time=[n/a] ; pid=8964 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2019-12-22 22:11:07 UTC", "InactiveExitTimestampMonotonic": "19300701", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "8964", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "network-online.target basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOn 2019-12-22T22:14:12.678 INFO:teuthology.task.ansible.out:ly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Sun 2019-12-22 22:11:07 UTC", "WatchdogTimestampMonotonic": "19300674", "WatchdogUSec": "0"}} 2019-12-22T22:14:12.731 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:12 +0000 (0:00:00.066) 0:02:35.909 ******* 2019-12-22T22:14:12.915 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.004452", "end": "2019-12-22 22:14:12.886669", "rc": 0, "start": "2019-12-22 22:14:12.882217", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2019-12-22T22:14:12.981 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:12 +0000 (0:00:00.250) 0:02:36.159 ******* 2019-12-22T22:14:13.020 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** fatal: [smithi043.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2019-12-22T22:14:13.060 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:13 +0000 (0:00:00.078) 0:02:36.238 ******* 2019-12-22T22:14:13.063 INFO:teuthology.task.ansible.out: 2019-12-22T22:14:13.083 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2019-12-22T22:14:13.109 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.004757", "end": "2019-12-22 22:14:13.057454", "rc": 0, "start": "2019-12-22 22:14:13.052697", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2019-12-22T22:14:13.142 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:13 +0000 (0:00:00.082) 0:02:36.320 ******* 2019-12-22T22:14:13.152 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:14:13.201 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:13 +0000 (0:00:00.055) 0:02:36.375 ******* 2019-12-22T22:14:13.505 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2019-12-22T22:14:13.505 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2019-12-22T22:14:13.549 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:13 +0000 (0:00:00.351) 0:02:36.727 ******* 2019-12-22T22:14:13.641 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2019-12-22T22:14:13.683 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:13 +0000 (0:00:00.130) 0:02:36.857 ******* 2019-12-22T22:14:14.228 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2019-12-22T22:14:14.229 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2019-12-22T22:14:14.271 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:14 +0000 (0:00:00.591) 0:02:37.449 ******* 2019-12-22T22:14:14.734 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2019-12-22T22:14:14.741 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2019-12-22 22:10:58 UTC", "ActiveEnterTimestampMonotonic": "22143883", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-journald.socket network.target time-sync.target var-run.mount local-fs.target system.slice basic.target nss-lookup.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Sun 2019-12-22 22:10:58 UTC", "AssertTimestampMonotonic": "22143349", "Before": "multi-user.target getty@tty1.service shutdown.target xdm.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2019-12-22 22:10:58 UTC", "ConditionTimestampMonotonic": "22143349", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8725", "ExecMainStartTimestamp": "Sun 2019-12-22 22:10:58 UTC", "ExecMainStartTimestampMonotonic": "22143827", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_err 2019-12-22T22:14:14.743 INFO:teuthology.task.ansible.out:ors=no ; start_time=[Sun 2019-12-22 22:10:58 UTC] ; stop_time=[n/a] ; pid=8725 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2019-12-22 22:10:58 UTC", "InactiveExitTimestampMonotonic": "22143883", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "8725", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "network-online.target basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOn 2019-12-22T22:14:14.744 INFO:teuthology.task.ansible.out:ly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Sun 2019-12-22 22:10:58 UTC", "WatchdogTimestampMonotonic": "22143862", "WatchdogUSec": "0"}} 2019-12-22T22:14:14.781 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:14 +0000 (0:00:00.509) 0:02:37.959 ******* 2019-12-22T22:14:15.120 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2019-12-22T22:14:15.121 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.004913", "end": "2019-12-22 22:14:15.086719", "rc": 0, "start": "2019-12-22 22:14:15.081806", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2019-12-22T22:14:15.163 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:15 +0000 (0:00:00.382) 0:02:38.341 ******* 2019-12-22T22:14:15.658 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2019-12-22T22:14:15.660 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2019-12-22T22:14:15.705 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:15 +0000 (0:00:00.541) 0:02:38.883 ******* 2019-12-22T22:14:39.732 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2019-12-22T22:14:39.736 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["selinux-policy-targeted", "3.13.1-252.el7_7.6.noarch from updates"]]}, "msg": "", "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.umd.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.fileplanet.com\n * updates: repos-va.psychz.net\nResolving Dependencies\n--> Running transaction check\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Running transaction check\n---> Package selinux-policy.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n selinux-policy-targeted noarch 3.13.1-252.el7_7.6 updates 7.0 M\nUpdating for dependencies:\n selinux-policy noarch 3.13.1-252.el7_7.6 updates 492 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package (+1 Dependent package)\n\nTotal download size: 7.5 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal 2019-12-22T22:14:39.738 INFO:teuthology.task.ansible.out: 26 MB/s | 7.5 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Updating : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Cleanup : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Cleanup : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n Verifying : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Verifying : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n\nUpdated:\n selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 \n\nDependency Updated:\n selinux-policy.noarch 0:3.13.1-252.el7_7.6 \n\nComplete!\n"]} 2019-12-22T22:14:39.782 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:39 +0000 (0:00:24.076) 0:03:02.959 ******* 2019-12-22T22:14:40.643 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2019-12-22T22:14:40.645 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2019-12-22T22:14:40.681 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:40 +0000 (0:00:00.899) 0:03:03.859 ******* 2019-12-22T22:14:41.191 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2019-12-22T22:14:41.192 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["selinux-policy-targeted", "3.13.1-252.el7_7.6.noarch from updates"]]}, "msg": "", "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.facebook.net\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.fileplanet.com\n * updates: repos-va.psychz.net\nResolving Dependencies\n--> Running transaction check\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Running transaction check\n---> Package selinux-policy.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n selinux-policy-targeted noarch 3.13.1-252.el7_7.6 updates 7.0 M\nUpdating for dependencies:\n selinux-policy noarch 3.13.1-252.el7_7.6 updates 492 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package (+1 Dependent package)\n\nTotal download size: 7.5 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal 2019-12-22T22:14:41.194 INFO:teuthology.task.ansible.out: 24 MB/s | 7.5 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Updating : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Cleanup : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Cleanup : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n Verifying : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Verifying : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n\nUpdated:\n selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 \n\nDependency Updated:\n selinux-policy.noarch 0:3.13.1-252.el7_7.6 \n\nComplete!\n"]} 2019-12-22T22:14:41.231 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:41 +0000 (0:00:00.549) 0:03:04.409 ******* 2019-12-22T22:14:41.794 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2019-12-22T22:14:41.832 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:41 +0000 (0:00:00.599) 0:03:05.008 ******* 2019-12-22T22:14:43.826 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2019-12-22T22:14:43.827 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["selinux-policy-targeted", "3.13.1-252.el7_7.6.noarch from updates"]]}, "msg": "", "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.umd.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.fileplanet.com\n * updates: repos-va.psychz.net\nResolving Dependencies\n--> Running transaction check\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Running transaction check\n---> Package selinux-policy.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n selinux-policy-targeted noarch 3.13.1-252.el7_7.6 updates 7.0 M\nUpdating for dependencies:\n selinux-policy noarch 3.13.1-252.el7_7.6 updates 492 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package (+1 Dependent package)\n\nTotal download size: 7.5 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal 2019-12-22T22:14:43.829 INFO:teuthology.task.ansible.out: 24 MB/s | 7.5 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Updating : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Cleanup : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Cleanup : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n Verifying : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Verifying : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n\nUpdated:\n selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 \n\nDependency Updated:\n selinux-policy.noarch 0:3.13.1-252.el7_7.6 \n\nComplete!\n"]} 2019-12-22T22:14:43.878 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:43 +0000 (0:00:02.047) 0:03:07.056 ******* 2019-12-22T22:14:44.427 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2019-12-22T22:14:44.429 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2019-12-22T22:14:44.465 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:44 +0000 (0:00:00.587) 0:03:07.643 ******* 2019-12-22T22:14:48.708 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:07.717089", "end": "2019-12-22 22:14:48.666362", "failed_when_result": false, "rc": 0, "start": "2019-12-22 22:14:40.949273", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2019-12-22T22:14:48.745 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:48 +0000 (0:00:04.280) 0:03:11.923 ******* 2019-12-22T22:14:49.628 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2019-12-22T22:14:49.630 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2019-12-22T22:14:49.667 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:49 +0000 (0:00:00.921) 0:03:12.844 ******* 2019-12-22T22:14:49.960 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.020211", "end": "2019-12-22 22:14:49.935145", "rc": 0, "start": "2019-12-22 22:14:49.914934", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2019-12-22T22:14:50.001 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:49 +0000 (0:00:00.334) 0:03:13.179 ******* 2019-12-22T22:14:50.097 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2019-12-22T22:14:50.099 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:07.934745", "end": "2019-12-22 22:14:50.068971", "failed_when_result": false, "rc": 0, "start": "2019-12-22 22:14:42.134226", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2019-12-22T22:14:50.140 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:50 +0000 (0:00:00.139) 0:03:13.318 ******* 2019-12-22T22:14:50.347 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.013504", "end": "2019-12-22 22:14:50.318987", "rc": 0, "start": "2019-12-22 22:14:50.305483", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:14:50.388 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:50 +0000 (0:00:00.247) 0:03:13.566 ******* 2019-12-22T22:14:51.115 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2019-12-22T22:14:51.164 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:51 +0000 (0:00:00.771) 0:03:14.338 ******* 2019-12-22T22:14:51.502 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.026930", "end": "2019-12-22 22:14:51.477905", "rc": 0, "start": "2019-12-22 22:14:51.450975", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2019-12-22T22:14:51.541 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:51 +0000 (0:00:00.379) 0:03:14.718 ******* 2019-12-22T22:14:51.903 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.017719", "end": "2019-12-22 22:14:51.867796", "rc": 0, "start": "2019-12-22 22:14:51.850077", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:14:51.949 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:51 +0000 (0:00:00.400) 0:03:15.118 ******* 2019-12-22T22:14:53.254 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2019-12-22T22:14:53.256 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:08.464656", "end": "2019-12-22 22:14:53.218666", "failed_when_result": false, "rc": 0, "start": "2019-12-22 22:14:44.754010", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2019-12-22T22:14:53.301 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:53 +0000 (0:00:01.360) 0:03:16.479 ******* 2019-12-22T22:14:54.203 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2019-12-22T22:14:54.238 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:54 +0000 (0:00:00.936) 0:03:17.416 ******* 2019-12-22T22:14:54.602 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2019-12-22T22:14:54.603 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.039897", "end": "2019-12-22 22:14:54.572748", "rc": 0, "start": "2019-12-22 22:14:54.532851", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2019-12-22T22:14:54.649 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:54 +0000 (0:00:00.411) 0:03:17.827 ******* 2019-12-22T22:14:55.003 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2019-12-22T22:14:55.003 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.018219", "end": "2019-12-22 22:14:54.978494", "rc": 0, "start": "2019-12-22 22:14:54.960275", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:14:55.043 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:55 +0000 (0:00:00.394) 0:03:18.221 ******* 2019-12-22T22:14:58.452 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:07.730880", "end": "2019-12-22 22:14:58.405576", "rc": 0, "start": "2019-12-22 22:14:50.674696", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:14:58.506 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:58 +0000 (0:00:03.462) 0:03:21.684 ******* 2019-12-22T22:14:58.864 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2019-12-22T22:14:58.919 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:58 +0000 (0:00:00.413) 0:03:22.097 ******* 2019-12-22T22:14:58.946 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2019-12-22T22:14:58.947 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi", "iozone"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2019-12-22T22:14:58.987 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:58 +0000 (0:00:00.067) 0:03:22.165 ******* 2019-12-22T22:14:59.025 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2019-12-22T22:14:59.027 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-22T22:14:59.070 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:59 +0000 (0:00:00.076) 0:03:22.241 ******* 2019-12-22T22:14:59.119 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** ok: [smithi091.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN", "python3"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2019-12-22T22:14:59.155 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:59 +0000 (0:00:00.091) 0:03:22.333 ******* 2019-12-22T22:14:59.212 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ ok: [smithi091.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-22T22:14:59.245 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:59 +0000 (0:00:00.089) 0:03:22.423 ******* 2019-12-22T22:14:59.588 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2019-12-22T22:14:59.589 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-22T22:14:59.639 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:59 +0000 (0:00:00.394) 0:03:22.817 ******* 2019-12-22T22:14:59.947 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "name": "ubuntu", "state": "present", "system": false} 2019-12-22T22:14:59.996 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:14:59 +0000 (0:00:00.354) 0:03:23.171 ******* 2019-12-22T22:15:00.033 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:00.079 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:00 +0000 (0:00:00.085) 0:03:23.256 ******* 2019-12-22T22:15:00.108 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2019-12-22T22:15:00.110 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:00.151 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:00 +0000 (0:00:00.067) 0:03:23.324 ******* 2019-12-22T22:15:00.444 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2019-12-22T22:15:00.445 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:08.157447", "end": "2019-12-22 22:15:00.397815", "rc": 0, "start": "2019-12-22 22:14:52.240368", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:15:00.495 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:00 +0000 (0:00:00.348) 0:03:23.672 ******* 2019-12-22T22:15:00.532 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2019-12-22T22:15:00.534 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2019-12-22T22:15:00.573 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:00 +0000 (0:00:00.078) 0:03:23.751 ******* 2019-12-22T22:15:00.829 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2019-12-22T22:15:00.830 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2019-12-22T22:15:00.868 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:00 +0000 (0:00:00.295) 0:03:24.046 ******* 2019-12-22T22:15:00.909 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2019-12-22T22:15:00.910 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi", "iozone"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2019-12-22T22:15:00.951 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:00 +0000 (0:00:00.079) 0:03:24.126 ******* 2019-12-22T22:15:00.985 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2019-12-22T22:15:00.986 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-22T22:15:01.033 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:01 +0000 (0:00:00.079) 0:03:24.205 ******* 2019-12-22T22:15:01.065 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2019-12-22T22:15:01.066 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN", "python3"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2019-12-22T22:15:01.105 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:01 +0000 (0:00:00.077) 0:03:24.283 ******* 2019-12-22T22:15:01.142 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2019-12-22T22:15:01.144 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-22T22:15:01.185 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:01 +0000 (0:00:00.079) 0:03:24.362 ******* 2019-12-22T22:15:01.529 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2019-12-22T22:15:01.531 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-22T22:15:01.576 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:01 +0000 (0:00:00.391) 0:03:24.754 ******* 2019-12-22T22:15:01.593 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2019-12-22T22:15:01.595 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2019-12-22T22:15:01.633 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:01 +0000 (0:00:00.056) 0:03:24.811 ******* 2019-12-22T22:15:01.671 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi091"}, "changed": false} 2019-12-22T22:15:01.721 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:01 +0000 (0:00:00.087) 0:03:24.899 ******* 2019-12-22T22:15:01.899 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2019-12-22T22:15:01.900 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "name": "ubuntu", "state": "present", "system": false} 2019-12-22T22:15:01.942 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:01 +0000 (0:00:00.217) 0:03:25.117 ******* 2019-12-22T22:15:01.977 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2019-12-22T22:15:01.979 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:02.020 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:02 +0000 (0:00:00.081) 0:03:25.198 ******* 2019-12-22T22:15:02.048 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2019-12-22T22:15:02.049 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:02.086 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:02 +0000 (0:00:00.065) 0:03:25.264 ******* 2019-12-22T22:15:02.445 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** ok: [smithi107.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2019-12-22T22:15:02.491 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:02 +0000 (0:00:00.401) 0:03:25.665 ******* 2019-12-22T22:15:02.757 INFO:teuthology.task.ansible.out: 2019-12-22T22:15:02.757 INFO:teuthology.task.ansible.out:TASK [testnode : Set the system's hostname] ************************************ 2019-12-22T22:15:02.759 INFO:teuthology.task.ansible.out: changed: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi091.front.sepia.ceph.com", "ansible_hostname": "smithi091", "ansible_nodename": "smithi091"}, "changed": true, "name": "smithi091"} 2019-12-22T22:15:02.807 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:02 +0000 (0:00:00.319) 0:03:25.985 ******* 2019-12-22T22:15:02.809 INFO:teuthology.task.ansible.out: 2019-12-22T22:15:03.533 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2019-12-22T22:15:03.577 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:03 +0000 (0:00:00.766) 0:03:26.751 ******* 2019-12-22T22:15:03.611 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* ok: [smithi107.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi107"}, "changed": false} 2019-12-22T22:15:03.655 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:03 +0000 (0:00:00.077) 0:03:26.829 ******* 2019-12-22T22:15:03.760 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2019-12-22T22:15:03.814 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:03 +0000 (0:00:00.162) 0:03:26.991 ******* 2019-12-22T22:15:03.985 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2019-12-22T22:15:03.985 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:08.593176", "end": "2019-12-22 22:15:03.947559", "rc": 0, "start": "2019-12-22 22:14:55.354383", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:15:04.023 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:04 +0000 (0:00:00.209) 0:03:27.201 ******* 2019-12-22T22:15:04.205 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2019-12-22T22:15:04.206 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi107.front.sepia.ceph.com", "ansible_hostname": "smithi107", "ansible_nodename": "smithi107"}, "changed": true, "name": "smithi107"} 2019-12-22T22:15:04.250 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:04 +0000 (0:00:00.227) 0:03:27.428 ******* 2019-12-22T22:15:04.394 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2019-12-22T22:15:04.396 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2019-12-22T22:15:04.455 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:04 +0000 (0:00:00.204) 0:03:27.633 ******* 2019-12-22T22:15:04.494 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2019-12-22T22:15:04.496 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi", "iozone"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2019-12-22T22:15:04.533 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:04 +0000 (0:00:00.076) 0:03:27.710 ******* 2019-12-22T22:15:04.580 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2019-12-22T22:15:04.581 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-22T22:15:04.638 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:04 +0000 (0:00:00.105) 0:03:27.815 ******* 2019-12-22T22:15:04.691 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2019-12-22T22:15:04.703 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN", "python3"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2019-12-22T22:15:04.743 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:04 +0000 (0:00:00.105) 0:03:27.921 ******* 2019-12-22T22:15:04.772 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2019-12-22T22:15:04.773 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2019-12-22T22:15:04.820 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:04 +0000 (0:00:00.076) 0:03:27.997 ******* 2019-12-22T22:15:04.838 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ ok: [smithi043.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-22T22:15:04.869 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:04 +0000 (0:00:00.049) 0:03:28.047 ******* 2019-12-22T22:15:05.180 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2019-12-22T22:15:05.239 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:05 +0000 (0:00:00.369) 0:03:28.417 ******* 2019-12-22T22:15:05.245 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2019-12-22T22:15:05.246 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-22T22:15:05.292 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:05 +0000 (0:00:00.051) 0:03:28.469 ******* 2019-12-22T22:15:05.623 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "name": "ubuntu", "state": "present", "system": false} 2019-12-22T22:15:05.668 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:05 +0000 (0:00:00.374) 0:03:28.843 ******* 2019-12-22T22:15:05.707 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:05.760 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:05 +0000 (0:00:00.089) 0:03:28.933 ******* 2019-12-22T22:15:05.765 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2019-12-22T22:15:05.768 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2019-12-22T22:15:05.808 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:05 +0000 (0:00:00.050) 0:03:28.983 ******* 2019-12-22T22:15:05.814 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:05.856 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:05 +0000 (0:00:00.048) 0:03:29.031 ******* 2019-12-22T22:15:06.199 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2019-12-22T22:15:06.253 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:06 +0000 (0:00:00.399) 0:03:29.431 ******* 2019-12-22T22:15:06.260 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2019-12-22T22:15:06.261 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2019-12-22T22:15:06.300 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:06 +0000 (0:00:00.046) 0:03:29.478 ******* 2019-12-22T22:15:06.809 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2019-12-22T22:15:06.847 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:06 +0000 (0:00:00.547) 0:03:30.025 ******* 2019-12-22T22:15:07.133 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2019-12-22T22:15:07.134 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2019-12-22T22:15:07.186 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:07 +0000 (0:00:00.338) 0:03:30.364 ******* 2019-12-22T22:15:07.206 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-22T22:15:07.207 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-22T22:15:07.250 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:07 +0000 (0:00:00.063) 0:03:30.428 ******* 2019-12-22T22:15:07.344 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2019-12-22T22:15:07.345 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2019-12-22T22:15:07.393 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:07 +0000 (0:00:00.142) 0:03:30.570 ******* 2019-12-22T22:15:07.427 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2019-12-22T22:15:07.429 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi043"}, "changed": false} 2019-12-22T22:15:07.473 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:07 +0000 (0:00:00.080) 0:03:30.651 ******* 2019-12-22T22:15:07.601 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "gid": 10103, "name": "kvm", "state": "present", "system": false} 2019-12-22T22:15:07.639 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:07 +0000 (0:00:00.165) 0:03:30.817 ******* 2019-12-22T22:15:07.640 INFO:teuthology.task.ansible.out: 2019-12-22T22:15:08.005 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-22T22:15:08.006 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1215, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2019-12-22T22:15:08.056 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:08 +0000 (0:00:00.413) 0:03:31.231 ******* 2019-12-22T22:15:08.061 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2019-12-22T22:15:08.065 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi043.front.sepia.ceph.com", "ansible_hostname": "smithi043", "ansible_nodename": "smithi043"}, "changed": true, "name": "smithi043"} 2019-12-22T22:15:08.107 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:08 +0000 (0:00:00.052) 0:03:31.283 ******* 2019-12-22T22:15:08.176 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2019-12-22T22:15:08.221 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:08 +0000 (0:00:00.115) 0:03:31.399 ******* 2019-12-22T22:15:08.552 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-22T22:15:08.554 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-22T22:15:08.606 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:08 +0000 (0:00:00.384) 0:03:31.784 ******* 2019-12-22T22:15:08.926 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-12-22T22:15:08.926 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "gid": 10103, "name": "kvm", "state": "present", "system": false} 2019-12-22T22:15:08.968 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:08 +0000 (0:00:00.362) 0:03:32.146 ******* 2019-12-22T22:15:09.019 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-12-22T22:15:09.019 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2019-12-22T22:15:09.064 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:09 +0000 (0:00:00.095) 0:03:32.242 ******* 2019-12-22T22:15:09.088 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2019-12-22T22:15:09.124 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:09 +0000 (0:00:00.054) 0:03:32.296 ******* 2019-12-22T22:15:09.310 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-22T22:15:09.311 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1215, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2019-12-22T22:15:09.378 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:09 +0000 (0:00:00.259) 0:03:32.556 ******* 2019-12-22T22:15:09.975 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-22T22:15:09.976 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2019-12-22T22:15:10.015 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:10 +0000 (0:00:00.636) 0:03:33.192 ******* 2019-12-22T22:15:10.084 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2019-12-22T22:15:10.085 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2019-12-22T22:15:10.129 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:10 +0000 (0:00:00.114) 0:03:33.307 ******* 2019-12-22T22:15:10.346 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-12-22T22:15:10.347 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2019-12-22T22:15:10.383 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:10 +0000 (0:00:00.254) 0:03:33.561 ******* 2019-12-22T22:15:11.051 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2019-12-22T22:15:11.052 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2019-12-22T22:15:11.110 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:11 +0000 (0:00:00.717) 0:03:34.279 ******* 2019-12-22T22:15:11.328 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-22T22:15:11.329 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2019-12-22T22:15:11.379 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:11 +0000 (0:00:00.277) 0:03:34.557 ******* 2019-12-22T22:15:12.058 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2019-12-22T22:15:12.059 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2019-12-22T22:15:12.109 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:12 +0000 (0:00:00.729) 0:03:35.286 ******* 2019-12-22T22:15:12.443 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-22T22:15:12.446 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.085340", "end": "2019-12-22 22:15:12.407969", "rc": 0, "start": "2019-12-22 22:15:10.322629", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:15:12.492 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:12 +0000 (0:00:00.383) 0:03:35.670 ******* 2019-12-22T22:15:12.500 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-22T22:15:12.505 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-22T22:15:12.541 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:12 +0000 (0:00:00.049) 0:03:35.719 ******* 2019-12-22T22:15:12.548 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2019-12-22T22:15:12.596 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:12 +0000 (0:00:00.048) 0:03:35.768 ******* 2019-12-22T22:15:12.884 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "gid": 10103, "name": "kvm", "state": "present", "system": false} 2019-12-22T22:15:12.924 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:12 +0000 (0:00:00.330) 0:03:36.099 ******* 2019-12-22T22:15:13.279 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-22T22:15:13.281 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1215, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2019-12-22T22:15:13.317 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:13 +0000 (0:00:00.395) 0:03:36.495 ******* 2019-12-22T22:15:13.593 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2019-12-22T22:15:13.594 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'CentOS 7 Local fastcgi Repo', u'baseurl': u'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/'}, 'key': u'centos7-fcgi-ceph'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2019-12-22T22:15:13.944 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-22T22:15:13.945 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.257418", "end": "2019-12-22 22:15:13.917752", "rc": 0, "start": "2019-12-22 22:15:11.660334", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:15:13.986 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:13 +0000 (0:00:00.669) 0:03:37.164 ******* 2019-12-22T22:15:14.015 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2019-12-22T22:15:14.057 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:14 +0000 (0:00:00.070) 0:03:37.235 ******* 2019-12-22T22:15:14.315 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-12-22T22:15:14.316 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2019-12-22T22:15:14.357 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:14 +0000 (0:00:00.299) 0:03:37.535 ******* 2019-12-22T22:15:14.432 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** ok: [smithi091.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'lab-extras', u'baseurl': u'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/'}, 'key': u'lab-extras'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2019-12-22T22:15:14.482 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:14 +0000 (0:00:00.125) 0:03:37.660 ******* 2019-12-22T22:15:14.510 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2019-12-22T22:15:14.553 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:14 +0000 (0:00:00.069) 0:03:37.730 ******* 2019-12-22T22:15:14.581 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2019-12-22T22:15:14.583 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:14.623 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:14 +0000 (0:00:00.070) 0:03:37.800 ******* 2019-12-22T22:15:14.664 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2019-12-22T22:15:14.666 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-22T22:15:14.668 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-12-22T22:15:14.712 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:14 +0000 (0:00:00.089) 0:03:37.890 ******* 2019-12-22T22:15:14.740 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2019-12-22T22:15:14.741 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:14.781 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:14 +0000 (0:00:00.069) 0:03:37.959 ******* 2019-12-22T22:15:14.820 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-22T22:15:14.822 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2019-12-22T22:15:14.882 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:14 +0000 (0:00:00.100) 0:03:38.060 ******* 2019-12-22T22:15:14.943 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2019-12-22T22:15:14.943 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'CentOS 7 Local fastcgi Repo', u'baseurl': u'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/'}, 'key': u'centos7-fcgi-ceph'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2019-12-22T22:15:15.307 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2019-12-22T22:15:15.369 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:15 +0000 (0:00:00.486) 0:03:38.547 ******* 2019-12-22T22:15:15.412 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2019-12-22T22:15:15.445 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:15 +0000 (0:00:00.075) 0:03:38.623 ******* 2019-12-22T22:15:15.845 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2019-12-22T22:15:15.847 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'lab-extras', u'baseurl': u'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/'}, 'key': u'lab-extras'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2019-12-22T22:15:15.887 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:15 +0000 (0:00:00.441) 0:03:39.064 ******* 2019-12-22T22:15:15.925 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2019-12-22T22:15:15.967 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:15 +0000 (0:00:00.078) 0:03:39.143 ******* 2019-12-22T22:15:15.997 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:16.038 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:16 +0000 (0:00:00.069) 0:03:39.212 ******* 2019-12-22T22:15:16.043 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.264851", "end": "2019-12-22 22:15:16.006668", "rc": 0, "start": "2019-12-22 22:15:15.741817", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.umd.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.fileplanet.com\n * updates: repos-va.psychz.net\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: mirror.umd.edu", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirror.fileplanet.com", " * updates: repos-va.psychz.net", "No unfinished transactions left."]} 2019-12-22T22:15:16.097 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:16 +0000 (0:00:00.062) 0:03:39.275 ******* 2019-12-22T22:15:16.105 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2019-12-22T22:15:16.105 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-22T22:15:16.107 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-12-22T22:15:16.147 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:16 +0000 (0:00:00.050) 0:03:39.325 ******* 2019-12-22T22:15:16.191 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2019-12-22T22:15:16.193 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:16.231 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:16 +0000 (0:00:00.083) 0:03:39.409 ******* 2019-12-22T22:15:16.269 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-22T22:15:16.271 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2019-12-22T22:15:16.308 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:16 +0000 (0:00:00.077) 0:03:39.486 ******* 2019-12-22T22:15:16.501 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2019-12-22T22:15:16.503 INFO:teuthology.task.ansible.out:fatal: [smithi091.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.055614", "end": "2019-12-22 22:15:16.462898", "msg": "non-zero return code", "rc": 1, "start": "2019-12-22 22:15:16.407284", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2019-12-22T22:15:16.556 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:16 +0000 (0:00:00.245) 0:03:39.732 ******* 2019-12-22T22:15:16.600 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2019-12-22T22:15:16.602 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:16.637 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:16 +0000 (0:00:00.083) 0:03:39.815 ******* 2019-12-22T22:15:16.803 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2019-12-22T22:15:16.804 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2019-12-22T22:15:16.852 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:16 +0000 (0:00:00.214) 0:03:40.030 ******* 2019-12-22T22:15:17.494 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.296015", "end": "2019-12-22 22:15:17.457295", "rc": 0, "start": "2019-12-22 22:15:17.161280", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.facebook.net\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.fileplanet.com\n * updates: repos-va.psychz.net\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: mirror.facebook.net", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirror.fileplanet.com", " * updates: repos-va.psychz.net", "No unfinished transactions left."]} 2019-12-22T22:15:17.555 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:17 +0000 (0:00:00.702) 0:03:40.733 ******* 2019-12-22T22:15:17.939 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-22T22:15:17.940 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.246686", "end": "2019-12-22 22:15:17.906197", "rc": 0, "start": "2019-12-22 22:15:15.659511", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:15:17.979 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:17 +0000 (0:00:00.423) 0:03:41.156 ******* 2019-12-22T22:15:17.985 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2019-12-22T22:15:17.986 INFO:teuthology.task.ansible.out:fatal: [smithi107.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.053558", "end": "2019-12-22 22:15:17.928134", "msg": "non-zero return code", "rc": 1, "start": "2019-12-22 22:15:17.874576", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2019-12-22T22:15:18.024 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:18 +0000 (0:00:00.045) 0:03:41.201 ******* 2019-12-22T22:15:18.030 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2019-12-22T22:15:18.068 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:18 +0000 (0:00:00.044) 0:03:41.246 ******* 2019-12-22T22:15:18.074 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2019-12-22T22:15:18.078 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:18.115 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:18 +0000 (0:00:00.046) 0:03:41.293 ******* 2019-12-22T22:15:18.997 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2019-12-22T22:15:18.998 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'CentOS 7 Local fastcgi Repo', u'baseurl': u'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/'}, 'key': u'centos7-fcgi-ceph'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2019-12-22T22:15:19.946 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'lab-extras', u'baseurl': u'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/'}, 'key': u'lab-extras'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2019-12-22T22:15:19.985 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:19 +0000 (0:00:01.870) 0:03:43.163 ******* 2019-12-22T22:15:20.023 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2019-12-22T22:15:20.062 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:20 +0000 (0:00:00.076) 0:03:43.240 ******* 2019-12-22T22:15:20.091 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:20.125 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:20 +0000 (0:00:00.063) 0:03:43.303 ******* 2019-12-22T22:15:20.163 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2019-12-22T22:15:20.164 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-22T22:15:20.177 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-12-22T22:15:20.213 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:20 +0000 (0:00:00.087) 0:03:43.391 ******* 2019-12-22T22:15:20.244 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2019-12-22T22:15:20.246 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:20.282 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:20 +0000 (0:00:00.069) 0:03:43.460 ******* 2019-12-22T22:15:20.320 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-22T22:15:20.322 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2019-12-22T22:15:20.359 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:20 +0000 (0:00:00.076) 0:03:43.537 ******* 2019-12-22T22:15:20.854 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2019-12-22T22:15:20.892 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:20 +0000 (0:00:00.532) 0:03:44.069 ******* 2019-12-22T22:15:21.573 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.315087", "end": "2019-12-22 22:15:21.532432", "rc": 0, "start": "2019-12-22 22:15:21.217345", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.umd.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.fileplanet.com\n * updates: repos-va.psychz.net\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: mirror.umd.edu", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirror.fileplanet.com", " * updates: repos-va.psychz.net", "No unfinished transactions left."]} 2019-12-22T22:15:21.625 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:21 +0000 (0:00:00.734) 0:03:44.803 ******* 2019-12-22T22:15:22.035 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2019-12-22T22:15:22.036 INFO:teuthology.task.ansible.out:fatal: [smithi043.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.052907", "end": "2019-12-22 22:15:21.997025", "msg": "non-zero return code", "rc": 1, "start": "2019-12-22 22:15:21.944118", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2019-12-22T22:15:22.079 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:22 +0000 (0:00:00.453) 0:03:45.257 ******* 2019-12-22T22:15:22.119 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:22.161 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:22 +0000 (0:00:00.082) 0:03:45.339 ******* 2019-12-22T22:15:25.214 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-12-22T22:15:25.215 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi", "iozone"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be erased\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n iozone x86_64 3.424-2_ceph.el7.centos @lab-extras 1.3 M\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 2019-12-22T22:15:25.216 INFO:teuthology.task.ansible.out: @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.1 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 4 Packages (+3 Dependent packages)\n\nInstalled size: 23 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 1/7 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 2/7 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 3/7 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/7 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/7 \n Erasing : iozone-3.424-2_ceph.el7.centos.x86_64 6/7 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/7 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/7 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/7 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 7/7 \n\nRemoved:\n iozone.x86_64 0:3.424-2_ceph.el7.centos 2019-12-22T22:15:25.218 INFO:teuthology.task.ansible.out: \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-22T22:15:25.252 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:25 +0000 (0:00:03.090) 0:03:48.429 ******* 2019-12-22T22:15:26.751 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi", "iozone"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be erased\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n iozone x86_64 3.424-2_ceph.el7.centos @lab-extras 1.3 M\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 2019-12-22T22:15:26.752 INFO:teuthology.task.ansible.out: @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.1 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 4 Packages (+3 Dependent packages)\n\nInstalled size: 23 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 1/7 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 2/7 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 3/7 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/7 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/7 \n Erasing : iozone-3.424-2_ceph.el7.centos.x86_64 6/7 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/7 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/7 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/7 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 7/7 \n\nRemoved:\n iozone.x86_64 0:3.424-2_ceph.el7.centos 2019-12-22T22:15:26.754 INFO:teuthology.task.ansible.out: \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-22T22:15:26.789 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:26 +0000 (0:00:01.537) 0:03:49.967 ******* 2019-12-22T22:15:28.641 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-22T22:15:28.642 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2019-12-22T22:15:28.682 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:28 +0000 (0:00:01.891) 0:03:51.858 ******* 2019-12-22T22:15:30.187 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2019-12-22T22:15:30.226 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:30 +0000 (0:00:01.545) 0:03:53.404 ******* 2019-12-22T22:15:31.008 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-12-22T22:15:31.009 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi", "iozone"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be erased\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n iozone x86_64 3.424-2_ceph.el7.centos @lab-extras 1.3 M\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 2019-12-22T22:15:31.011 INFO:teuthology.task.ansible.out: @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.1 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 4 Packages (+3 Dependent packages)\n\nInstalled size: 23 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 1/7 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 2/7 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 3/7 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/7 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/7 \n Erasing : iozone-3.424-2_ceph.el7.centos.x86_64 6/7 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/7 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/7 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/7 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 7/7 \n\nRemoved:\n iozone.x86_64 0:3.424-2_ceph.el7.centos 2019-12-22T22:15:31.013 INFO:teuthology.task.ansible.out: \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-22T22:15:31.050 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:31 +0000 (0:00:00.823) 0:03:54.228 ******* 2019-12-22T22:15:34.555 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-22T22:15:34.556 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2019-12-22T22:15:34.594 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:34 +0000 (0:00:03.543) 0:03:57.771 ******* 2019-12-22T22:15:52.085 INFO:teuthology.task.ansible.out: 2019-12-22T22:15:52.087 INFO:teuthology.task.ansible.out:TASK [testnode : Install packages] ********************************************* 2019-12-22T22:15:52.088 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": ["@core", "@base", "iozone", "qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "python3"]}, "msg": "Warning: Group core does not have any packages to install.\nWarning: Group base does not have any packages to install.\nGroup base does have 1 conditional packages, which may get installed.\n", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-20.el7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7.x86_64 providing 2019-12-22T22:15:52.089 INFO:teuthology.task.ansible.out:libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is already installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-open 2019-12-22T22:15:52.091 INFO:teuthology.task.ansible.out:jdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already installed", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.umd.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.fileplanet.com\n * updates: repos-va.psychz.net\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be installed\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package python3.x86_64 0:3.6.8-10.el7 will be installed\n--> Processing Dependency: python3-libs(x86-64) = 3.6.8-10.el7 for package: python3-3.6.8-10.el7.x86_64\n--> Processing Dependency: python3-setuptools for package: python3-3.6.8-10.el7.x86_64\n--> Processing Dependency: python3-pip for package: python3-3.6.8-10.el7.x86_64\n--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-10.el7.x86_64\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Running transaction check\n---> Package li 2019-12-22T22:15:52.093 INFO:teuthology.task.ansible.out:brados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package python3-libs.x86_64 0:3.6.8-10.el7 will be installed\n---> Package python3-pip.noarch 0:9.0.3-5.el7 will be installed\n---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n iozone x86_64 3.424-2_ceph.el7.centos lab-extras 800 k\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n python3 x86_64 3.6.8-10.el7 base 69 k\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-27.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n python3-libs x86_64 3.6.8-10.el7 base 7.0 M\n python3-pip noarch 9.0.3-5.el7 base 1.8 M\n python3-setuptools noarch 39.2.0-10.el7 base 629 k\n qemu-img x86_64 10:1.5.3-167.el7_7.1 updates 699 k\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 updates 435 k\n\nTra 2019-12-22T22:15:52.095 INFO:teuthology.task.ansible.out:nsaction Summary\n================================================================================\nInstall 4 Packages (+8 Dependent packages)\n\nTotal download size: 18 M\nInstalled size: 70 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 33 MB/s | 18 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-27.el7.x86_64 1/12 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/12 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/12 \n Installing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 4/12 \n Installing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 5/12 \n Installing : python3-libs-3.6.8-10.el7.x86_64 6/12 \n Installing : python3-setuptools-39.2.0-10.el7.noarch 7/12 \n Installing : python3-3.6.8-10.el7.x86_64 8/12 \n Installing : python3-pip-9.0.3-5.el7.noarch 9/12 \n Installing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 10/12 \n Installing : iozone-3.424-2_ceph.el7.centos.x86_64 11/12 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 12/12 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/12 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/12 \n Verifying : python3-pip-9.0.3-5.el7.noarch 3/12 \n Verifying : python3-libs-3.6.8-10.el7.x86_64 4/12 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 5/12 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 6/12 \n Verifying : python3-setuptools-39.2. 2019-12-22T22:15:52.097 INFO:teuthology.task.ansible.out:0-10.el7.noarch 7/12 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 8/12 \n Verifying : boost-random-1.53.0-27.el7.x86_64 9/12 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 10/12 \n Verifying : python3-3.6.8-10.el7.x86_64 11/12 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 12/12 \n\nInstalled:\n iozone.x86_64 0:3.424-2_ceph.el7.centos \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n python3.x86_64 0:3.6.8-10.el7 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-27.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n python3-libs.x86_64 0:3.6.8-10.el7 \n python3-pip.noarch 0:9.0.3-5.el7 \n python3-setuptools.noarch 0:39.2.0-10.el7 \n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-22T22:15:52.144 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:52 +0000 (0:00:17.550) 0:04:15.322 ******* 2019-12-22T22:15:52.922 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2019-12-22T22:15:52.923 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2019-12-22T22:15:52.980 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:52 +0000 (0:00:00.836) 0:04:16.158 ******* 2019-12-22T22:15:53.553 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-12-22T22:15:53.554 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2019-12-22T22:15:53.612 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:53 +0000 (0:00:00.631) 0:04:16.790 ******* 2019-12-22T22:15:53.652 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-22T22:15:53.654 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:53.694 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:53 +0000 (0:00:00.082) 0:04:16.872 ******* 2019-12-22T22:15:53.823 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-22T22:15:53.825 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": ["@core", "@base", "iozone", "qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "python3"]}, "msg": "Warning: Group core does not have any packages to install.\nWarning: Group base does not have any packages to install.\nGroup base does have 1 conditional packages, which may get installed.\n", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-20.el7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-pro 2019-12-22T22:15:53.827 INFO:teuthology.task.ansible.out:gs is already installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is al 2019-12-22T22:15:53.829 INFO:teuthology.task.ansible.out:ready installed", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.facebook.net\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.fileplanet.com\n * updates: repos-va.psychz.net\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be installed\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package python3.x86_64 0:3.6.8-10.el7 will be installed\n--> Processing Dependency: python3-libs(x86-64) = 3.6.8-10.el7 for package: python3-3.6.8-10.el7.x86_64\n--> Processing Dependency: python3-setuptools for package: python3-3.6.8-10.el7.x86_64\n--> Processing Dependency: python3-pip for package: python3-3.6.8-10.el7.x86_64\n--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-10.el7.x86_64\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: l 2019-12-22T22:15:53.830 INFO:teuthology.task.ansible.out:ibboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package python3-libs.x86_64 0:3.6.8-10.el7 will be installed\n---> Package python3-pip.noarch 0:9.0.3-5.el7 will be installed\n---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n iozone x86_64 3.424-2_ceph.el7.centos lab-extras 800 k\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n python3 x86_64 3.6.8-10.el7 base 69 k\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-27.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n python3-libs x86_64 3.6.8-10.el7 base 7.0 M\n python3-pip noarch 9.0.3-5.el7 base 1.8 M\n python3-setuptools noarch 39.2.0-10.el7 base 629 k\n qemu-img x86_64 10:1.5.3-167.el7_7.1 updates 699 k\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 updates 435 k\n\nTransaction Summary\n=========================================================== 2019-12-22T22:15:53.832 INFO:teuthology.task.ansible.out:=====================\nInstall 4 Packages (+8 Dependent packages)\n\nTotal download size: 18 M\nInstalled size: 70 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 33 MB/s | 18 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-27.el7.x86_64 1/12 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/12 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/12 \n Installing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 4/12 \n Installing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 5/12 \n Installing : python3-libs-3.6.8-10.el7.x86_64 6/12 \n Installing : python3-setuptools-39.2.0-10.el7.noarch 7/12 \n Installing : python3-3.6.8-10.el7.x86_64 8/12 \n Installing : python3-pip-9.0.3-5.el7.noarch 9/12 \n Installing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 10/12 \n Installing : iozone-3.424-2_ceph.el7.centos.x86_64 11/12 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 12/12 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/12 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/12 \n Verifying : python3-pip-9.0.3-5.el7.noarch 3/12 \n Verifying : python3-libs-3.6.8-10.el7.x86_64 4/12 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 5/12 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 6/12 \n Verifying : python3-setuptools-39.2.0-10.el7.noarch 7/12 \n Verifying : 10:qemu-img-1.5.3-1 2019-12-22T22:15:53.833 INFO:teuthology.task.ansible.out:67.el7_7.1.x86_64 8/12 \n Verifying : boost-random-1.53.0-27.el7.x86_64 9/12 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 10/12 \n Verifying : python3-3.6.8-10.el7.x86_64 11/12 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 12/12 \n\nInstalled:\n iozone.x86_64 0:3.424-2_ceph.el7.centos \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n python3.x86_64 0:3.6.8-10.el7 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-27.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n python3-libs.x86_64 0:3.6.8-10.el7 \n python3-pip.noarch 0:9.0.3-5.el7 \n python3-setuptools.noarch 0:39.2.0-10.el7 \n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-22T22:15:53.878 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:53 +0000 (0:00:00.184) 0:04:17.056 ******* 2019-12-22T22:15:54.155 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2019-12-22T22:15:54.157 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "basic.target dbus.service system.slice polkit.service", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "shutdown.target ebtables.service ip6tables.service iptables.service ipset.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "Igno 2019-12-22T22:15:54.159 INFO:teuthology.task.ansible.out:reOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", " 2019-12-22T22:15:54.160 INFO:teuthology.task.ansible.out:TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "system.slice network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-22T22:15:54.199 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:54 +0000 (0:00:00.320) 0:04:17.377 ******* 2019-12-22T22:15:54.621 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2019-12-22T22:15:54.622 INFO:teuthology.task.ansible.out:fatal: [smithi091.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2019-12-22T22:15:54.667 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:54 +0000 (0:00:00.468) 0:04:17.845 ******* 2019-12-22T22:15:54.697 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2019-12-22T22:15:54.697 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2019-12-22T22:15:54.742 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:54 +0000 (0:00:00.074) 0:04:17.920 ******* 2019-12-22T22:15:55.267 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-12-22T22:15:55.268 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2019-12-22T22:15:55.308 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:55 +0000 (0:00:00.566) 0:04:18.486 ******* 2019-12-22T22:15:55.346 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-22T22:15:55.348 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:55.387 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:55 +0000 (0:00:00.078) 0:04:18.565 ******* 2019-12-22T22:15:55.567 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2019-12-22T22:15:55.568 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2019-12-22T22:15:55.617 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:55 +0000 (0:00:00.229) 0:04:18.795 ******* 2019-12-22T22:15:55.645 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:55.693 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:55 +0000 (0:00:00.075) 0:04:18.870 ******* 2019-12-22T22:15:55.721 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2019-12-22T22:15:55.758 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:55 +0000 (0:00:00.065) 0:04:18.936 ******* 2019-12-22T22:15:55.789 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2019-12-22T22:15:55.797 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:55.827 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:55 +0000 (0:00:00.068) 0:04:19.005 ******* 2019-12-22T22:15:55.834 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2019-12-22T22:15:55.836 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "polkit.service dbus.service system.slice basic.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "ip6tables.service ebtables.service iptables.service shutdown.target ipset.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "Igno 2019-12-22T22:15:55.838 INFO:teuthology.task.ansible.out:reOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127510", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127510", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", " 2019-12-22T22:15:55.840 INFO:teuthology.task.ansible.out:TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "network-pre.target system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-22T22:15:55.897 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:55 +0000 (0:00:00.069) 0:04:19.074 ******* 2019-12-22T22:15:55.905 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2019-12-22T22:15:55.907 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:55.947 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:55 +0000 (0:00:00.050) 0:04:19.125 ******* 2019-12-22T22:15:55.999 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2019-12-22T22:15:56.001 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-12-22T22:15:56.012 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-22T22:15:56.060 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:56 +0000 (0:00:00.111) 0:04:19.236 ******* 2019-12-22T22:15:56.089 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2019-12-22T22:15:56.105 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:56.129 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:56 +0000 (0:00:00.070) 0:04:19.307 ******* 2019-12-22T22:15:56.158 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2019-12-22T22:15:56.197 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:56 +0000 (0:00:00.067) 0:04:19.375 ******* 2019-12-22T22:15:56.227 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2019-12-22T22:15:56.229 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:56.269 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:56 +0000 (0:00:00.071) 0:04:19.447 ******* 2019-12-22T22:15:56.299 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2019-12-22T22:15:56.301 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:56.338 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:56 +0000 (0:00:00.068) 0:04:19.516 ******* 2019-12-22T22:15:56.347 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2019-12-22T22:15:56.348 INFO:teuthology.task.ansible.out:fatal: [smithi107.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2019-12-22T22:15:56.416 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:56 +0000 (0:00:00.078) 0:04:19.594 ******* 2019-12-22T22:15:56.423 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:56.462 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:56 +0000 (0:00:00.046) 0:04:19.640 ******* 2019-12-22T22:15:56.490 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-22T22:15:56.508 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:56.528 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:56 +0000 (0:00:00.065) 0:04:19.706 ******* 2019-12-22T22:15:56.555 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2019-12-22T22:15:56.557 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:56.593 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:56 +0000 (0:00:00.065) 0:04:19.771 ******* 2019-12-22T22:15:56.621 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-22T22:15:56.622 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:56.667 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:56 +0000 (0:00:00.074) 0:04:19.845 ******* 2019-12-22T22:15:56.696 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:56.732 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:56 +0000 (0:00:00.064) 0:04:19.910 ******* 2019-12-22T22:15:56.759 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2019-12-22T22:15:56.761 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:56.800 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:56 +0000 (0:00:00.068) 0:04:19.978 ******* 2019-12-22T22:15:56.806 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2019-12-22T22:15:56.807 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2019-12-22T22:15:56.862 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:56 +0000 (0:00:00.045) 0:04:20.023 ******* TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:56.891 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:56 +0000 (0:00:00.045) 0:04:20.069 ******* 2019-12-22T22:15:56.898 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2019-12-22T22:15:56.900 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:56.946 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:56 +0000 (0:00:00.055) 0:04:20.124 ******* 2019-12-22T22:15:56.952 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2019-12-22T22:15:56.954 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:56.996 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:56 +0000 (0:00:00.049) 0:04:20.174 ******* 2019-12-22T22:15:57.005 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2019-12-22T22:15:57.066 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.069) 0:04:20.243 ******* 2019-12-22T22:15:57.075 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2019-12-22T22:15:57.079 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:57.128 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.055) 0:04:20.299 ******* 2019-12-22T22:15:57.136 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:57.169 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.047) 0:04:20.347 ******* 2019-12-22T22:15:57.175 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2019-12-22T22:15:57.175 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:57.214 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.044) 0:04:20.392 ******* 2019-12-22T22:15:57.221 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2019-12-22T22:15:57.222 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:57.260 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.045) 0:04:20.438 ******* 2019-12-22T22:15:57.266 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2019-12-22T22:15:57.267 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:57.308 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.048) 0:04:20.486 ******* 2019-12-22T22:15:57.316 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* skipping: [smithi107.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-12-22T22:15:57.317 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-22T22:15:57.357 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.048) 0:04:20.535 ******* 2019-12-22T22:15:57.363 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2019-12-22T22:15:57.364 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:57.405 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.047) 0:04:20.583 ******* 2019-12-22T22:15:57.410 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2019-12-22T22:15:57.416 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:57.456 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.051) 0:04:20.634 ******* 2019-12-22T22:15:57.462 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-22T22:15:57.463 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:57.505 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.048) 0:04:20.683 ******* 2019-12-22T22:15:57.512 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2019-12-22T22:15:57.552 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.047) 0:04:20.730 ******* 2019-12-22T22:15:57.558 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-12-22T22:15:57.560 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:57.601 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.048) 0:04:20.779 ******* 2019-12-22T22:15:57.607 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2019-12-22T22:15:57.608 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:57.649 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.047) 0:04:20.827 ******* 2019-12-22T22:15:57.654 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-22T22:15:57.657 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:57.699 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.049) 0:04:20.876 ******* 2019-12-22T22:15:57.712 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:57.753 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.046) 0:04:20.923 ******* TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:57.794 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.048) 0:04:20.972 ******* 2019-12-22T22:15:57.800 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2019-12-22T22:15:57.802 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:57.844 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.046) 0:04:21.018 ******* 2019-12-22T22:15:57.845 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-22T22:15:57.846 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:57.886 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.045) 0:04:21.063 ******* 2019-12-22T22:15:57.894 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:57.932 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.046) 0:04:21.110 ******* 2019-12-22T22:15:57.939 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-22T22:15:57.941 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:57.978 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:57 +0000 (0:00:00.045) 0:04:21.156 ******* 2019-12-22T22:15:57.984 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2019-12-22T22:15:57.985 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:58.024 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:58 +0000 (0:00:00.045) 0:04:21.201 ******* 2019-12-22T22:15:58.041 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-12-22T22:15:58.043 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:58.082 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:58 +0000 (0:00:00.057) 0:04:21.259 ******* 2019-12-22T22:15:58.087 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-22T22:15:58.089 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:58.126 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:58 +0000 (0:00:00.044) 0:04:21.303 ******* 2019-12-22T22:15:58.131 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-22T22:15:58.131 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:58.169 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:58 +0000 (0:00:00.043) 0:04:21.347 ******* 2019-12-22T22:15:58.175 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2019-12-22T22:15:58.177 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:58.214 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:58 +0000 (0:00:00.044) 0:04:21.392 ******* 2019-12-22T22:15:58.222 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-12-22T22:15:58.224 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:58.259 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:58 +0000 (0:00:00.044) 0:04:21.437 ******* 2019-12-22T22:15:58.265 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2019-12-22T22:15:58.267 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:58.303 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:58 +0000 (0:00:00.044) 0:04:21.481 ******* 2019-12-22T22:15:58.321 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:58.355 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:58 +0000 (0:00:00.051) 0:04:21.533 ******* 2019-12-22T22:15:58.363 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:58.405 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:58 +0000 (0:00:00.050) 0:04:21.583 ******* 2019-12-22T22:15:58.411 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-22T22:15:58.413 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:58.454 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:58 +0000 (0:00:00.046) 0:04:21.630 ******* 2019-12-22T22:15:58.461 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2019-12-22T22:15:58.462 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:58.500 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:58 +0000 (0:00:00.047) 0:04:21.677 ******* 2019-12-22T22:15:58.505 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-22T22:15:58.507 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:58.544 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:58 +0000 (0:00:00.044) 0:04:21.722 ******* 2019-12-22T22:15:58.551 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:58.590 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:58 +0000 (0:00:00.045) 0:04:21.768 ******* 2019-12-22T22:15:58.618 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:58.657 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:58 +0000 (0:00:00.066) 0:04:21.835 ******* 2019-12-22T22:15:58.689 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:58.723 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:58 +0000 (0:00:00.065) 0:04:21.901 ******* 2019-12-22T22:15:58.750 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2019-12-22T22:15:58.752 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:58.787 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:58 +0000 (0:00:00.064) 0:04:21.965 ******* 2019-12-22T22:15:58.836 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-22T22:15:58.838 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:58.877 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:58 +0000 (0:00:00.089) 0:04:22.054 ******* 2019-12-22T22:15:58.883 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-22T22:15:58.884 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2019-12-22T22:15:58.924 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:58 +0000 (0:00:00.047) 0:04:22.102 ******* 2019-12-22T22:15:58.934 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:58.970 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:58 +0000 (0:00:00.046) 0:04:22.148 ******* 2019-12-22T22:15:59.010 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-22T22:15:59.012 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:59.047 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:59 +0000 (0:00:00.076) 0:04:22.225 ******* 2019-12-22T22:15:59.076 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:59.127 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:59 +0000 (0:00:00.075) 0:04:22.300 ******* 2019-12-22T22:15:59.149 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-22T22:15:59.151 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:59.186 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:59 +0000 (0:00:00.063) 0:04:22.364 ******* 2019-12-22T22:15:59.214 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-22T22:15:59.216 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:59.257 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:59 +0000 (0:00:00.066) 0:04:22.430 ******* 2019-12-22T22:15:59.282 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-22T22:15:59.284 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-22T22:15:59.320 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:59 +0000 (0:00:00.067) 0:04:22.498 ******* 2019-12-22T22:15:59.322 INFO:teuthology.task.ansible.out: 2019-12-22T22:15:59.328 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-12-22T22:15:59.331 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:59.373 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:59 +0000 (0:00:00.052) 0:04:22.550 ******* 2019-12-22T22:15:59.422 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-22T22:15:59.424 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:59.470 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:59 +0000 (0:00:00.090) 0:04:22.641 ******* TASK [testnode : Install packages] ********************************************* ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": ["@core", "@base", "iozone", "qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "python3"]}, "msg": "Warning: Group core does not have any packages to install.\nWarning: Group base does not have any packages to install.\nGroup base does have 1 conditional packages, which may get installed.\n", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-20.el7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_ 2019-12-22T22:15:59.481 INFO:teuthology.task.ansible.out:64 providing openssl is already installed", "libuuid-2.23.2-61.el7.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is already installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig 2019-12-22T22:15:59.482 INFO:teuthology.task.ansible.out:-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already installed", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.umd.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.fileplanet.com\n * updates: repos-va.psychz.net\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be installed\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package python3.x86_64 0:3.6.8-10.el7 will be installed\n--> Processing Dependency: python3-libs(x86-64) = 3.6.8-10.el7 for package: python3-3.6.8-10.el7.x86_64\n--> Processing Dependency: python3-setuptools for package: python3-3.6.8-10.el7.x86_64\n--> Processing Dependency: python3-pip for package: python3-3.6.8-10.el7.x86_64\n--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-10.el7.x86_64\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 2019-12-22T22:15:59.484 INFO:teuthology.task.ansible.out:10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package python3-libs.x86_64 0:3.6.8-10.el7 will be installed\n---> Package python3-pip.noarch 0:9.0.3-5.el7 will be installed\n---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n iozone x86_64 3.424-2_ceph.el7.centos lab-extras 800 k\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n python3 x86_64 3.6.8-10.el7 base 69 k\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-27.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n python3-libs x86_64 3.6.8-10.el7 base 7.0 M\n python3-pip noarch 9.0.3-5.el7 base 1.8 M\n python3-setuptools noarch 39.2.0-10.el7 base 629 k\n qemu-img x86_64 10:1.5.3-167.el7_7.1 updates 699 k\n q 2019-12-22T22:15:59.486 INFO:teuthology.task.ansible.out:emu-kvm-common x86_64 10:1.5.3-167.el7_7.1 updates 435 k\n\nTransaction Summary\n================================================================================\nInstall 4 Packages (+8 Dependent packages)\n\nTotal download size: 18 M\nInstalled size: 70 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 30 MB/s | 18 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-27.el7.x86_64 1/12 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/12 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/12 \n Installing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 4/12 \n Installing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 5/12 \n Installing : python3-libs-3.6.8-10.el7.x86_64 6/12 \n Installing : python3-setuptools-39.2.0-10.el7.noarch 7/12 \n Installing : python3-3.6.8-10.el7.x86_64 8/12 \n Installing : python3-pip-9.0.3-5.el7.noarch 9/12 \n Installing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 10/12 \n Installing : iozone-3.424-2_ceph.el7.centos.x86_64 11/12 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 12/12 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/12 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/12 \n Verifying : python3-pip-9.0.3-5.el7.noarch 3/12 \n Verifying : python3-libs-3.6.8-10.el7.x86_64 4/12 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 5/12 \n Verifying : mod_fastcgi-2.4.7-1.ce 2019-12-22T22:15:59.492 INFO:teuthology.task.ansible.out:ph.el7.centos.x86_64 6/12 \n Verifying : python3-setuptools-39.2.0-10.el7.noarch 7/12 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 8/12 \n Verifying : boost-random-1.53.0-27.el7.x86_64 9/12 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 10/12 \n Verifying : python3-3.6.8-10.el7.x86_64 11/12 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 12/12 \n\nInstalled:\n iozone.x86_64 0:3.424-2_ceph.el7.centos \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n python3.x86_64 0:3.6.8-10.el7 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-27.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n python3-libs.x86_64 0:3.6.8-10.el7 \n python3-pip.noarch 0:9.0.3-5.el7 \n python3-setuptools.noarch 0:39.2.0-10.el7 \n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-22T22:15:59.520 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:59 +0000 (0:00:00.049) 0:04:22.690 ******* TASK [testnode : Remove packages] ********************************************** 2019-12-22T22:15:59.522 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:59.564 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:59 +0000 (0:00:00.050) 0:04:22.741 ******* 2019-12-22T22:15:59.593 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:59.660 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:59 +0000 (0:00:00.090) 0:04:22.832 ******* 2019-12-22T22:15:59.664 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-22T22:15:59.665 INFO:teuthology.task.ansible.out: [WARNING]: Cannot set fs attributes on a non-existent symlink target. follow should be set to False to avoid this. 2019-12-22T22:15:59.668 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2019-12-22T22:15:59.716 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:59 +0000 (0:00:00.056) 0:04:22.889 ******* 2019-12-22T22:15:59.720 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:59.757 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:59 +0000 (0:00:00.045) 0:04:22.935 ******* 2019-12-22T22:15:59.768 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:59.804 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:59 +0000 (0:00:00.047) 0:04:22.982 ******* 2019-12-22T22:15:59.810 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-22T22:15:59.812 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:59.854 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:59 +0000 (0:00:00.049) 0:04:23.032 ******* 2019-12-22T22:15:59.860 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-22T22:15:59.862 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:59.900 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:59 +0000 (0:00:00.046) 0:04:23.078 ******* 2019-12-22T22:15:59.928 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-22T22:15:59.929 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:15:59.965 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:15:59 +0000 (0:00:00.064) 0:04:23.143 ******* 2019-12-22T22:16:00.002 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-22T22:16:00.004 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:00.046 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:00 +0000 (0:00:00.081) 0:04:23.224 ******* 2019-12-22T22:16:00.075 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:00.112 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:00 +0000 (0:00:00.065) 0:04:23.290 ******* 2019-12-22T22:16:00.140 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-22T22:16:00.142 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:00.183 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:00 +0000 (0:00:00.070) 0:04:23.361 ******* 2019-12-22T22:16:00.199 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-22T22:16:00.200 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2019-12-22T22:16:00.237 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:00 +0000 (0:00:00.054) 0:04:23.415 ******* 2019-12-22T22:16:00.243 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-22T22:16:00.245 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:00.283 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:00 +0000 (0:00:00.045) 0:04:23.460 ******* 2019-12-22T22:16:00.310 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-22T22:16:00.312 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:00.353 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:00 +0000 (0:00:00.070) 0:04:23.531 ******* 2019-12-22T22:16:00.364 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2019-12-22T22:16:00.365 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2019-12-22T22:16:00.403 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:00 +0000 (0:00:00.049) 0:04:23.581 ******* 2019-12-22T22:16:00.409 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-22T22:16:00.411 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:00.447 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:00 +0000 (0:00:00.044) 0:04:23.625 ******* 2019-12-22T22:16:00.472 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2019-12-22T22:16:00.474 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:00.512 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:00 +0000 (0:00:00.064) 0:04:23.690 ******* 2019-12-22T22:16:00.540 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2019-12-22T22:16:00.543 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:00.580 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:00 +0000 (0:00:00.066) 0:04:23.756 ******* 2019-12-22T22:16:00.595 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-22T22:16:00.595 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-22T22:16:00.632 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:00 +0000 (0:00:00.054) 0:04:23.810 ******* 2019-12-22T22:16:00.638 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-22T22:16:00.639 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:00.677 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:00 +0000 (0:00:00.044) 0:04:23.855 ******* 2019-12-22T22:16:00.714 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-22T22:16:00.716 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:00.751 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:00 +0000 (0:00:00.073) 0:04:23.929 ******* 2019-12-22T22:16:00.778 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2019-12-22T22:16:00.780 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:00.820 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:00 +0000 (0:00:00.065) 0:04:23.994 ******* 2019-12-22T22:16:00.843 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2019-12-22T22:16:00.845 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:00.883 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:00 +0000 (0:00:00.066) 0:04:24.061 ******* 2019-12-22T22:16:00.915 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2019-12-22T22:16:00.917 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:00.963 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:00 +0000 (0:00:00.079) 0:04:24.141 ******* 2019-12-22T22:16:01.007 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} TASK [testnode : Make fsstress same path as ubuntu.] *************************** ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2019-12-22T22:16:01.011 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.048) 0:04:24.189 ******* 2019-12-22T22:16:01.048 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.036) 0:04:24.226 ******* 2019-12-22T22:16:01.053 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2019-12-22T22:16:01.055 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:01.103 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.054) 0:04:24.281 ******* 2019-12-22T22:16:01.139 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.036) 0:04:24.317 ******* 2019-12-22T22:16:01.147 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-22T22:16:01.149 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:01.189 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.049) 0:04:24.367 ******* 2019-12-22T22:16:01.204 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:01.235 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.046) 0:04:24.413 ******* 2019-12-22T22:16:01.245 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-22T22:16:01.247 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:01.285 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.049) 0:04:24.462 ******* 2019-12-22T22:16:01.291 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2019-12-22T22:16:01.292 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:01.330 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.045) 0:04:24.508 ******* 2019-12-22T22:16:01.336 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-22T22:16:01.337 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:01.375 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.044) 0:04:24.553 ******* 2019-12-22T22:16:01.385 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:01.419 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.044) 0:04:24.597 ******* 2019-12-22T22:16:01.426 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-22T22:16:01.428 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:01.465 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.045) 0:04:24.643 ******* 2019-12-22T22:16:01.471 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2019-12-22T22:16:01.475 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:01.516 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.047) 0:04:24.690 ******* 2019-12-22T22:16:01.519 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-22T22:16:01.520 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:01.562 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.048) 0:04:24.739 ******* 2019-12-22T22:16:01.575 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Stop and disable firewalld] *********************************** ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "polkit.service dbus.service system.slice basic.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "shutdown.target ebtables.service ipset.service iptables.service ip6tables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time= 2019-12-22T22:16:01.576 INFO:teuthology.task.ansible.out:[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000" 2019-12-22T22:16:01.578 INFO:teuthology.task.ansible.out:, "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "system.slice network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-22T22:16:01.609 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.047) 0:04:24.787 ******* 2019-12-22T22:16:01.644 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.034) 0:04:24.822 ******* 2019-12-22T22:16:01.650 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-22T22:16:01.652 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:01.696 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.051) 0:04:24.874 ******* 2019-12-22T22:16:01.702 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-22T22:16:01.704 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2019-12-22T22:16:01.751 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.054) 0:04:24.929 ******* 2019-12-22T22:16:01.757 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-22T22:16:01.759 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:01.803 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.050) 0:04:24.980 ******* 2019-12-22T22:16:01.824 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi091.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} ok: [smithi091.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} skipping: [smithi091.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:01.831 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:01.840 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi091.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:01.881 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.077) 0:04:25.058 ******* 2019-12-22T22:16:01.887 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:01.888 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:01.933 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.052) 0:04:25.111 ******* 2019-12-22T22:16:01.962 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:01.986 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:01 +0000 (0:00:00.052) 0:04:25.164 ******* 2019-12-22T22:16:02.035 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2019-12-22T22:16:02.038 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:02.093 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:02 +0000 (0:00:00.107) 0:04:25.271 ******* 2019-12-22T22:16:02.102 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2019-12-22T22:16:02.103 INFO:teuthology.task.ansible.out:fatal: [smithi043.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2019-12-22T22:16:02.147 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:02 +0000 (0:00:00.053) 0:04:25.325 ******* 2019-12-22T22:16:02.154 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2019-12-22T22:16:02.156 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for teuthology-lock command] **************************** 2019-12-22T22:16:02.156 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.005725", "end": "2019-12-22 22:16:02.103297", "rc": 0, "start": "2019-12-22 22:16:02.097572", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2019-12-22T22:16:02.200 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:02 +0000 (0:00:00.052) 0:04:25.378 ******* 2019-12-22T22:16:02.239 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:02 +0000 (0:00:00.039) 0:04:25.417 ******* 2019-12-22T22:16:02.246 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:02.290 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:02 +0000 (0:00:00.050) 0:04:25.467 ******* 2019-12-22T22:16:02.296 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-22T22:16:02.298 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:02.350 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:02 +0000 (0:00:00.061) 0:04:25.528 ******* 2019-12-22T22:16:02.357 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2019-12-22T22:16:02.359 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:02.405 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:02 +0000 (0:00:00.054) 0:04:25.582 ******* 2019-12-22T22:16:02.411 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-22T22:16:02.413 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:02.451 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:02 +0000 (0:00:00.046) 0:04:25.629 ******* 2019-12-22T22:16:02.457 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2019-12-22T22:16:02.459 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:02.503 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:02 +0000 (0:00:00.050) 0:04:25.680 ******* 2019-12-22T22:16:02.512 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:02.555 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:02 +0000 (0:00:00.052) 0:04:25.732 ******* 2019-12-22T22:16:02.562 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2019-12-22T22:16:02.563 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:02.564 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2019-12-22T22:16:02.609 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:02 +0000 (0:00:00.054) 0:04:25.787 ******* 2019-12-22T22:16:02.645 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:02 +0000 (0:00:00.035) 0:04:25.823 ******* 2019-12-22T22:16:02.650 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2019-12-22T22:16:02.651 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for custom repos] *************************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:02.690 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:02 +0000 (0:00:00.045) 0:04:25.868 ******* 2019-12-22T22:16:02.727 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:02 +0000 (0:00:00.036) 0:04:25.905 ******* 2019-12-22T22:16:02.732 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2019-12-22T22:16:02.777 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:02 +0000 (0:00:00.048) 0:04:25.953 ******* 2019-12-22T22:16:02.785 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2019-12-22T22:16:02.796 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:02.825 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:02 +0000 (0:00:00.049) 0:04:26.003 ******* 2019-12-22T22:16:02.831 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2019-12-22T22:16:02.833 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:02.871 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:02 +0000 (0:00:00.045) 0:04:26.048 ******* 2019-12-22T22:16:02.891 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:02.921 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:02 +0000 (0:00:00.050) 0:04:26.099 ******* 2019-12-22T22:16:02.927 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2019-12-22T22:16:02.929 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:02.972 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:02 +0000 (0:00:00.049) 0:04:26.148 ******* 2019-12-22T22:16:02.978 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:03.029 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.053) 0:04:26.202 ******* 2019-12-22T22:16:03.031 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* skipping: [smithi043.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi043.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:03.070 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.046) 0:04:26.248 ******* 2019-12-22T22:16:03.085 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:03.121 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.050) 0:04:26.299 ******* 2019-12-22T22:16:03.132 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:03.168 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.047) 0:04:26.346 ******* 2019-12-22T22:16:03.176 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2019-12-22T22:16:03.176 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2019-12-22T22:16:03.178 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:03.219 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.050) 0:04:26.396 ******* 2019-12-22T22:16:03.255 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.036) 0:04:26.433 ******* 2019-12-22T22:16:03.262 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2019-12-22T22:16:03.312 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.054) 0:04:26.487 ******* 2019-12-22T22:16:03.317 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ ok: [smithi091.front.sepia.ceph.com] => (item={u'block_used': 13037562, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227286922, u'size_available': 930967232512, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966956, u'device': u'/dev/sda1', u'inode_used': 88020, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227286922, "block_size": 4096, "block_total": 240324484, "block_used": 13037562, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966956, "inode_total": 61054976, "inode_used": 88020, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930967232512, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}} 2019-12-22T22:16:03.321 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:03.357 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.047) 0:04:26.534 ******* 2019-12-22T22:16:03.391 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.034) 0:04:26.569 ******* 2019-12-22T22:16:03.404 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:03.436 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.045) 0:04:26.614 ******* 2019-12-22T22:16:03.445 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2019-12-22T22:16:03.449 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:03.483 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.047) 0:04:26.661 ******* 2019-12-22T22:16:03.489 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2019-12-22T22:16:03.490 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:03.530 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.046) 0:04:26.708 ******* 2019-12-22T22:16:03.536 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-22T22:16:03.537 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2019-12-22T22:16:03.584 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.048) 0:04:26.756 ******* TASK [testnode : Ensure packages are not present.] ***************************** 2019-12-22T22:16:03.586 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:03.624 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.045) 0:04:26.802 ******* 2019-12-22T22:16:03.631 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2019-12-22T22:16:03.632 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:03.633 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} skipping: [smithi107.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:03.644 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:03.646 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:03.683 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.059) 0:04:26.861 ******* 2019-12-22T22:16:03.690 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2019-12-22T22:16:03.695 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi107.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi107.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:03.737 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.052) 0:04:26.914 ******* 2019-12-22T22:16:03.742 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:03.743 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2019-12-22T22:16:03.743 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.014280", "end": "2019-12-22 22:16:03.688623", "rc": 0, "start": "2019-12-22 22:16:03.674343", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:03.786 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.047) 0:04:26.961 ******* 2019-12-22T22:16:03.820 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.036) 0:04:26.998 ******* 2019-12-22T22:16:03.828 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:03.868 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.048) 0:04:27.046 ******* 2019-12-22T22:16:03.875 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2019-12-22T22:16:03.876 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={u'block_used': 13037562, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227286922, u'size_available': 930967232512, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966956, u'device': u'/dev/sda1', u'inode_used': 88020, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227286922, "block_size": 4096, "block_total": 240324484, "block_used": 13037562, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966956, "inode_total": 61054976, "inode_used": 88020, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930967232512, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:03.916 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.046) 0:04:27.093 ******* 2019-12-22T22:16:03.923 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2019-12-22T22:16:03.925 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.008039", "end": "2019-12-22 22:16:03.865832", "rc": 0, "start": "2019-12-22 22:16:03.857793", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} TASK [testnode : Install non aarch64 packages] ********************************* skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:03.967 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:03 +0000 (0:00:00.048) 0:04:27.142 ******* 2019-12-22T22:16:04.016 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:04 +0000 (0:00:00.051) 0:04:27.194 ******* 2019-12-22T22:16:04.023 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2019-12-22T22:16:04.037 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:04.065 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:04 +0000 (0:00:00.049) 0:04:27.243 ******* 2019-12-22T22:16:04.100 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:04 +0000 (0:00:00.034) 0:04:27.277 ******* 2019-12-22T22:16:04.106 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2019-12-22T22:16:04.110 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:04.111 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:04.154 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:04 +0000 (0:00:00.054) 0:04:27.332 ******* 2019-12-22T22:16:04.204 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:04 +0000 (0:00:00.041) 0:04:27.374 ******* TASK [testnode : Zap all non-root disks] *************************************** 2019-12-22T22:16:04.206 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5L78A', u'wwn-0x5000c500918dacd0'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5L78A', u'wwn': u'0x5000c500918dacd0', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5L78A-part1', u'wwn-0x5000c500918dacd0-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L78A", "wwn-0x5000c500918dacd0"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L78A-part1", "wwn-0x5000c500918dacd0-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5L78A", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-22T22:16:04.208 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c500918dacd0"}}, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure fuse group exists.] ************************************ skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:04.245 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:04 +0000 (0:00:00.049) 0:04:27.423 ******* 2019-12-22T22:16:04.252 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2019-12-22T22:16:04.254 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:04.292 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:04 +0000 (0:00:00.046) 0:04:27.470 ******* 2019-12-22T22:16:04.301 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2019-12-22T22:16:04.303 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:04.338 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:04 +0000 (0:00:00.045) 0:04:27.516 ******* 2019-12-22T22:16:04.344 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2019-12-22T22:16:04.346 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:04.384 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:04 +0000 (0:00:00.046) 0:04:27.562 ******* 2019-12-22T22:16:04.390 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2019-12-22T22:16:04.392 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:04.434 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:04 +0000 (0:00:00.044) 0:04:27.607 ******* 2019-12-22T22:16:04.437 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:04.476 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:04 +0000 (0:00:00.047) 0:04:27.654 ******* 2019-12-22T22:16:04.487 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2019-12-22T22:16:04.487 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:04.520 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:04 +0000 (0:00:00.043) 0:04:27.698 ******* 2019-12-22T22:16:04.547 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2019-12-22T22:16:04.549 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:04.586 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:04 +0000 (0:00:00.065) 0:04:27.764 ******* 2019-12-22T22:16:04.613 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-22T22:16:04.615 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:04.656 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:04 +0000 (0:00:00.066) 0:04:27.831 ******* 2019-12-22T22:16:04.683 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-12-22T22:16:04.685 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:04.721 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:04 +0000 (0:00:00.067) 0:04:27.898 ******* 2019-12-22T22:16:04.762 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-22T22:16:04.764 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:04.797 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:04 +0000 (0:00:00.076) 0:04:27.975 ******* 2019-12-22T22:16:04.835 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-12-22T22:16:04.843 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:04.878 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:04 +0000 (0:00:00.080) 0:04:28.056 ******* 2019-12-22T22:16:04.905 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-22T22:16:04.907 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:04.954 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:04 +0000 (0:00:00.075) 0:04:28.132 ******* 2019-12-22T22:16:04.966 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2019-12-22T22:16:04.967 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2019-12-22T22:16:05.012 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:05 +0000 (0:00:00.058) 0:04:28.190 ******* 2019-12-22T22:16:05.019 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-22T22:16:05.019 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:05.067 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:05 +0000 (0:00:00.054) 0:04:28.244 ******* 2019-12-22T22:16:05.076 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ ok: [smithi107.front.sepia.ceph.com] => (item={u'block_used': 13037585, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227286899, u'size_available': 930967138304, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966956, u'device': u'/dev/sda1', u'inode_used': 88020, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227286899, "block_size": 4096, "block_total": 240324484, "block_used": 13037585, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966956, "inode_total": 61054976, "inode_used": 88020, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930967138304, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}} 2019-12-22T22:16:05.125 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:05 +0000 (0:00:00.058) 0:04:28.303 ******* 2019-12-22T22:16:05.131 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-12-22T22:16:05.132 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:05.177 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:05 +0000 (0:00:00.051) 0:04:28.355 ******* 2019-12-22T22:16:05.215 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-22T22:16:05.217 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:05.257 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:05 +0000 (0:00:00.080) 0:04:28.435 ******* 2019-12-22T22:16:05.291 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:05.325 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:05 +0000 (0:00:00.067) 0:04:28.502 ******* 2019-12-22T22:16:05.351 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-22T22:16:05.353 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:05.395 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:05 +0000 (0:00:00.064) 0:04:28.567 ******* 2019-12-22T22:16:05.416 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-22T22:16:05.418 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:05.466 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:05 +0000 (0:00:00.077) 0:04:28.644 ******* 2019-12-22T22:16:05.472 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2019-12-22T22:16:05.473 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.011887", "end": "2019-12-22 22:16:05.413740", "rc": 0, "start": "2019-12-22 22:16:05.401853", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:05.512 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:05 +0000 (0:00:00.045) 0:04:28.690 ******* 2019-12-22T22:16:05.517 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-22T22:16:05.518 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:05.562 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:05 +0000 (0:00:00.050) 0:04:28.740 ******* 2019-12-22T22:16:05.569 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2019-12-22T22:16:05.570 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={u'block_used': 13037585, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227286899, u'size_available': 930967138304, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966956, u'device': u'/dev/sda1', u'inode_used': 88020, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227286899, "block_size": 4096, "block_total": 240324484, "block_used": 13037585, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966956, "inode_total": 61054976, "inode_used": 88020, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930967138304, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:05.611 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:05 +0000 (0:00:00.046) 0:04:28.787 ******* 2019-12-22T22:16:05.637 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2019-12-22T22:16:05.675 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:05 +0000 (0:00:00.065) 0:04:28.853 ******* 2019-12-22T22:16:05.714 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-22T22:16:05.715 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5R0J2', u'wwn-0x5000c50091e2f7f6'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5R0J2', u'wwn': u'0x5000c50091e2f7f6', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5R0J2-part1', u'wwn-0x5000c50091e2f7f6-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5R0J2", "wwn-0x5000c50091e2f7f6"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5R0J2-part1", "wwn-0x5000c50091e2f7f6-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5R0J2", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-22T22:16:05.717 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c50091e2f7f6"}}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:05.894 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-22T22:16:05.896 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2019-12-22T22:16:05.932 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:05 +0000 (0:00:00.256) 0:04:29.110 ******* 2019-12-22T22:16:06.284 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-22T22:16:06.286 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-22T22:16:06.323 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:06 +0000 (0:00:00.390) 0:04:29.500 ******* 2019-12-22T22:16:06.531 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-22T22:16:06.533 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-WiHycH-rlGc-QAE9-MInI-PyWN-ZtC7-cWh39j', u'nvme-INTEL_SSDPEDMD400G4_PHFT620400Q6400BGN', u'nvme-nvme.8086-50484654363230343030513634303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'PHFT620400Q6400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:02.099810", "end": "2019-12-22 22:16:06.493371", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-WiHycH-rlGc-QAE9-MInI-PyWN-ZtC7-cWh39j", "nvme-INTEL_SSDPEDMD400G4_PHFT620400Q6400BGN", "nvme-nvme.8086-50484654363230343030513634303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "PHFT620400Q6400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:04.393 2019-12-22T22:16:06.538 INFO:teuthology.task.ansible.out:561", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-22T22:16:06.655 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-22T22:16:06.657 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2019-12-22T22:16:06.700 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:06 +0000 (0:00:00.370) 0:04:29.871 ******* 2019-12-22T22:16:06.732 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-22T22:16:06.734 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:06.781 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:06 +0000 (0:00:00.076) 0:04:29.948 ******* 2019-12-22T22:16:06.799 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-22T22:16:06.801 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:06.839 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:06 +0000 (0:00:00.069) 0:04:30.017 ******* 2019-12-22T22:16:06.885 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-22T22:16:06.886 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:06.941 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:06 +0000 (0:00:00.101) 0:04:30.119 ******* 2019-12-22T22:16:06.969 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-22T22:16:06.971 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:07.017 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:07 +0000 (0:00:00.069) 0:04:30.188 ******* 2019-12-22T22:16:07.040 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:07.079 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:07 +0000 (0:00:00.068) 0:04:30.257 ******* 2019-12-22T22:16:07.118 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-22T22:16:07.120 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:07.181 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:07 +0000 (0:00:00.099) 0:04:30.357 ******* 2019-12-22T22:16:07.212 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-22T22:16:07.214 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:07.251 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:07 +0000 (0:00:00.070) 0:04:30.428 ******* 2019-12-22T22:16:07.278 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-22T22:16:07.281 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:07.329 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:07 +0000 (0:00:00.078) 0:04:30.506 ******* 2019-12-22T22:16:07.357 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-22T22:16:07.359 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:07.403 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:07 +0000 (0:00:00.074) 0:04:30.580 ******* 2019-12-22T22:16:07.441 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2019-12-22T22:16:07.444 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:07.478 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:07 +0000 (0:00:00.075) 0:04:30.656 ******* 2019-12-22T22:16:07.516 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2019-12-22T22:16:07.523 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:07.566 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:07 +0000 (0:00:00.088) 0:04:30.744 ******* 2019-12-22T22:16:07.595 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-22T22:16:07.597 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:07.639 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:07 +0000 (0:00:00.072) 0:04:30.817 ******* 2019-12-22T22:16:07.678 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-22T22:16:07.680 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:07.721 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:07 +0000 (0:00:00.081) 0:04:30.898 ******* 2019-12-22T22:16:07.748 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2019-12-22T22:16:07.750 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:07.786 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:07 +0000 (0:00:00.065) 0:04:30.964 ******* 2019-12-22T22:16:07.826 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2019-12-22T22:16:07.828 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:07.864 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:07 +0000 (0:00:00.078) 0:04:31.042 ******* 2019-12-22T22:16:07.882 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-22T22:16:07.884 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-O32All-jdM5-8UnR-3zk2-wWAb-VP6E-Vij1G5', u'nvme-INTEL_SSDPEDMD400G4_PHFT620400NK400BGN', u'nvme-nvme.8086-504846543632303430304e4b34303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'PHFT620400NK400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.847635", "end": "2019-12-22 22:16:07.850150", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-O32All-jdM5-8UnR-3zk2-wWAb-VP6E-Vij1G5", "nvme-INTEL_SSDPEDMD400G4_PHFT620400NK400BGN", "nvme-nvme.8086-504846543632303430304e4b34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "PHFT620400NK400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:06.002 2019-12-22T22:16:07.885 INFO:teuthology.task.ansible.out:515", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-22T22:16:07.897 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccUxColh13A1Rg2ZC76Zv1f22h51NnjU05R'], u'uuids': [u'8996ca0b-331c-4446-a848-1142b89f44f3']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400Q6400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.068174", "end": "2019-12-22 22:16:07.857072", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccUxColh13A1Rg2ZC76Zv1f22h51NnjU05R"], "labels": [], "masters": [], "uuids": ["8996ca0b-331c-4446-a848-1142b89f44f3"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620400Q6400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:06.788898", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-22T22:16:07.899 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:07.940 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:07 +0000 (0:00:00.076) 0:04:31.118 ******* 2019-12-22T22:16:07.990 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2019-12-22T22:16:07.992 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:08.032 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:08 +0000 (0:00:00.092) 0:04:31.210 ******* 2019-12-22T22:16:08.059 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2019-12-22T22:16:08.061 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:08.109 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:08 +0000 (0:00:00.076) 0:04:31.287 ******* 2019-12-22T22:16:08.136 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:08.174 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:08 +0000 (0:00:00.063) 0:04:31.350 ******* 2019-12-22T22:16:08.199 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2019-12-22T22:16:08.201 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:08.238 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:08 +0000 (0:00:00.065) 0:04:31.416 ******* 2019-12-22T22:16:08.267 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:08.301 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:08 +0000 (0:00:00.063) 0:04:31.479 ******* 2019-12-22T22:16:08.329 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2019-12-22T22:16:08.331 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:08.383 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:08 +0000 (0:00:00.080) 0:04:31.560 ******* 2019-12-22T22:16:08.426 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-22T22:16:08.428 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2019-12-22T22:16:08.463 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:08 +0000 (0:00:00.079) 0:04:31.639 ******* 2019-12-22T22:16:08.513 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi043.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:08.522 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2019-12-22T22:16:08.533 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:08.555 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi043.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:08.566 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:08.568 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:08.614 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:08 +0000 (0:00:00.152) 0:04:31.792 ******* 2019-12-22T22:16:08.778 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi043.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.005025", "end": "2019-12-22 22:16:08.746901", "rc": 0, "start": "2019-12-22 22:16:08.741876", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2019-12-22T22:16:08.824 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:08 +0000 (0:00:00.209) 0:04:32.001 ******* 2019-12-22T22:16:08.862 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2019-12-22T22:16:08.864 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:08.910 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:08 +0000 (0:00:00.086) 0:04:32.088 ******* 2019-12-22T22:16:08.949 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2019-12-22T22:16:08.950 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:08.987 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:08 +0000 (0:00:00.077) 0:04:32.165 ******* 2019-12-22T22:16:09.025 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:09.068 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:09 +0000 (0:00:00.076) 0:04:32.242 ******* 2019-12-22T22:16:09.092 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:09.131 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:09 +0000 (0:00:00.066) 0:04:32.309 ******* 2019-12-22T22:16:09.279 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-22T22:16:09.281 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNIWBINC6y79LorvAaeof2qlHGDe2M56r3'], u'uuids': [u'2617102f-34dd-4a6f-9157-aad2c316a072']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400NK400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.075275", "end": "2019-12-22 22:16:09.241839", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNIWBINC6y79LorvAaeof2qlHGDe2M56r3"], "labels": [], "masters": [], "uuids": ["2617102f-34dd-4a6f-9157-aad2c316a072"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620400NK400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:08.166564", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-22T22:16:09.317 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccU5bHeCe0AfRapUtIDEG5MhJoc93aRxcX0'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400Q6400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.066274", "end": "2019-12-22 22:16:09.281617", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccU5bHeCe0AfRapUtIDEG5MhJoc93aRxcX0"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620400Q6400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:08.215343", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-22T22:16:09.630 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2019-12-22T22:16:09.672 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:09 +0000 (0:00:00.540) 0:04:32.850 ******* 2019-12-22T22:16:09.723 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-12-22T22:16:09.725 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item={u'block_used': 13037598, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227286886, u'size_available': 930967085056, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966952, u'device': u'/dev/sda1', u'inode_used': 88024, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227286886, "block_size": 4096, "block_total": 240324484, "block_used": 13037598, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966952, "inode_total": 61054976, "inode_used": 88024, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930967085056, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}} 2019-12-22T22:16:09.762 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:09 +0000 (0:00:00.090) 0:04:32.940 ******* 2019-12-22T22:16:10.117 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2019-12-22T22:16:10.120 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.013790", "end": "2019-12-22 22:16:10.085199", "rc": 0, "start": "2019-12-22 22:16:10.071409", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:10.179 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:10 +0000 (0:00:00.416) 0:04:33.356 ******* 2019-12-22T22:16:10.229 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2019-12-22T22:16:10.230 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={u'block_used': 13037598, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227286886, u'size_available': 930967085056, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966952, u'device': u'/dev/sda1', u'inode_used': 88024, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227286886, "block_size": 4096, "block_total": 240324484, "block_used": 13037598, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966952, "inode_total": 61054976, "inode_used": 88024, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930967085056, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:10.276 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:10 +0000 (0:00:00.097) 0:04:33.454 ******* 2019-12-22T22:16:10.356 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2019-12-22T22:16:10.386 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:10 +0000 (0:00:00.110) 0:04:33.564 ******* 2019-12-22T22:16:10.449 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi043.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4HQ2G', u'wwn-0x5000c500871a8b6f'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W4HQ2G', u'wwn': u'0x5000c500871a8b6f', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4HQ2G-part1', u'wwn-0x5000c500871a8b6f-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQ2G", "wwn-0x5000c500871a8b6f"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQ2G-part1", "wwn-0x5000c500871a8b6f-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": " 2019-12-22T22:16:10.453 INFO:teuthology.task.ansible.out:512", "serial": "Z1W4HQ2G", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871a8b6f"}}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:10.683 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccUeNxh5W3AyVvfWk62Sc2jusDXjtphF8kh'], u'uuids': [u'850cf4a6-ebb7-407e-b7c9-04585a3e9e96']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400Q6400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.060862", "end": "2019-12-22 22:16:10.648274", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccUeNxh5W3AyVvfWk62Sc2jusDXjtphF8kh"], "labels": [], "masters": [], "uuids": ["850cf4a6-ebb7-407e-b7c9-04585a3e9e96"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620400Q6400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:09.587412", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-22T22:16:10.697 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNXKTPlQpPFE2QYkBrqXGrW4J2A0X3w9SK'], u'uuids': [u'85136b6c-451f-4f69-8d97-3873db09767c']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400NK400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.096805", "end": "2019-12-22 22:16:10.662798", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNXKTPlQpPFE2QYkBrqXGrW4J2A0X3w9SK"], "labels": [], "masters": [], "uuids": ["85136b6c-451f-4f69-8d97-3873db09767c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620400NK400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:09.565993", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-22T22:16:12.095 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNQuC9vSdq1miI2lBCNk7WDqxI0SLZwExq'], u'uuids': [u'3af6d14a-1a2a-4a31-8e4a-b8fce9400610']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400NK400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.076501", "end": "2019-12-22 22:16:12.058620", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNQuC9vSdq1miI2lBCNk7WDqxI0SLZwExq"], "labels": [], "masters": [], "uuids": ["3af6d14a-1a2a-4a31-8e4a-b8fce9400610"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620400NK400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:10.982119", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-22T22:16:12.101 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccUyubYCCe66p7fhec857ekxRWepm57mwqE'], u'uuids': [u'dff2aee7-addf-4709-93c8-ac4d39d9a6b1']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400Q6400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.070958", "end": "2019-12-22 22:16:12.058444", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccUyubYCCe66p7fhec857ekxRWepm57mwqE"], "labels": [], "masters": [], "uuids": ["dff2aee7-addf-4709-93c8-ac4d39d9a6b1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "PHFT620400Q6400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:10.987486", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-22T22:16:12.640 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-0dkcpz-0Fzx-92HN-SU1I-Ptpx-J3VN-djPj3F', u'nvme-INTEL_SSDPED1D480GA_PHMB751300GH480DGN', u'nvme-nvme.8086-50484d42373531333030474834383044474e-494e54454c2053534450454431443438304741-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', u'support_discard': u'512', u'serial': u'PHMB751300GH480DGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'447.13 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'937703088', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPED1D480GA', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.876177", "end": "2019-12-22 22:16:12.601631", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-0dkcpz-0Fzx-92HN-SU1I-Ptpx-J3VN-djPj3F", "nvme-INTEL_SSDPED1D480GA_PHMB751300GH480DGN", "nvme-nvme.8086-50484d42373531333030474834383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "serial": "PHMB751300GH480DGN", "size": "447.13 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:10.725454", "stderr" 2019-12-22T22:16:12.642 INFO:teuthology.task.ansible.out:: "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-22T22:16:13.512 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccUgKJCcCegdUHAuHCufS36nCU18NB3imhS'], u'uuids': [u'856ebd99-7c67-476c-9a99-7ca0483e9ef0']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400Q6400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.094540", "end": "2019-12-22 22:16:13.474486", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccUgKJCcCegdUHAuHCufS36nCU18NB3imhS"], "labels": [], "masters": [], "uuids": ["856ebd99-7c67-476c-9a99-7ca0483e9ef0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620400Q6400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:12.379946", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-22T22:16:13.538 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNWpc6j19MtDGBPJXOgsRkKWCay0wBfPKY'], u'uuids': [u'774c2207-8848-40f0-89a5-45715203e380']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400NK400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.099306", "end": "2019-12-22 22:16:13.499020", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNWpc6j19MtDGBPJXOgsRkKWCay0wBfPKY"], "labels": [], "masters": [], "uuids": ["774c2207-8848-40f0-89a5-45715203e380"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "PHFT620400NK400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:12.399714", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-22T22:16:13.604 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:13 +0000 (0:00:03.217) 0:04:36.782 ******* 2019-12-22T22:16:13.658 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-22T22:16:13.660 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5L78A', u'wwn-0x5000c500918dacd0'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5L78A', u'wwn': u'0x5000c500918dacd0', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5L78A-part1', u'wwn-0x5000c500918dacd0-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L78A", "wwn-0x5000c500918dacd0"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L78A-part1", "wwn-0x5000c500918dacd0-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5L78A", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-22T22:16:13.662 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c500918dacd0"}}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:14.007 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-22T22:16:14.008 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2hOn7Lm3o9lo3sFoocQs18lTgzPje6X6O'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHMB751300GH480DGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.046372", "end": "2019-12-22 22:16:13.965056", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2hOn7Lm3o9lo3sFoocQs18lTgzPje6X6O"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHMB751300GH480DGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:12.918684", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-22T22:16:14.031 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-22T22:16:14.033 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-WiHycH-rlGc-QAE9-MInI-PyWN-ZtC7-cWh39j', u'nvme-INTEL_SSDPEDMD400G4_PHFT620400Q6400BGN', u'nvme-nvme.8086-50484654363230343030513634303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'PHFT620400Q6400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.081636", "end": "2019-12-22 22:16:13.989779", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-WiHycH-rlGc-QAE9-MInI-PyWN-ZtC7-cWh39j", "nvme-INTEL_SSDPEDMD400G4_PHFT620400Q6400BGN", "nvme-nvme.8086-50484654363230343030513634303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "PHFT620400Q6400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:13.908143", "stderr": 2019-12-22T22:16:14.034 INFO:teuthology.task.ansible.out: "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0751397 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0751397 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:14.479 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccUxColh13A1Rg2ZC76Zv1f22h51NnjU05R'], u'uuids': [u'8996ca0b-331c-4446-a848-1142b89f44f3']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400Q6400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.184876", "end": "2019-12-22 22:16:14.443463", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccUxColh13A1Rg2ZC76Zv1f22h51NnjU05R"], "labels": [], "masters": [], "uuids": ["8996ca0b-331c-4446-a848-1142b89f44f3"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620400Q6400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:14.258587", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.178408 s, 647 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.178408 s, 647 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:14.921 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-22T22:16:14.924 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNGoy2T1idwrObPPgrWb2bbFCjByE3ImPA'], u'uuids': [u'252c6067-b9f0-4b05-8f26-cb1a1dae4f6f']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400NK400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.072956", "end": "2019-12-22 22:16:14.882236", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNGoy2T1idwrObPPgrWb2bbFCjByE3ImPA"], "labels": [], "masters": [], "uuids": ["252c6067-b9f0-4b05-8f26-cb1a1dae4f6f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620400NK400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:13.809280", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-22T22:16:14.974 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:14 +0000 (0:00:01.369) 0:04:38.152 ******* 2019-12-22T22:16:14.981 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-22T22:16:14.984 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccU5bHeCe0AfRapUtIDEG5MhJoc93aRxcX0'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400Q6400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.201339", "end": "2019-12-22 22:16:14.933713", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccU5bHeCe0AfRapUtIDEG5MhJoc93aRxcX0"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620400Q6400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:14.732374", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.194815 s, 592 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.194815 s, 592 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:15.018 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5R0J2', u'wwn-0x5000c50091e2f7f6'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5R0J2', u'wwn': u'0x5000c50091e2f7f6', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5R0J2-part1', u'wwn-0x5000c50091e2f7f6-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5R0J2", "wwn-0x5000c50091e2f7f6"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5R0J2-part1", "wwn-0x5000c50091e2f7f6-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5R0J2", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-22T22:16:15.020 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c50091e2f7f6"}}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:15.373 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-22T22:16:15.374 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2q23HoDfYqzA2b3jG4EziUhKfSIW7SmQJ'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHMB751300GH480DGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.044870", "end": "2019-12-22 22:16:15.340541", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2q23HoDfYqzA2b3jG4EziUhKfSIW7SmQJ"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHMB751300GH480DGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:14.295671", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-22T22:16:15.428 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-22T22:16:15.429 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-O32All-jdM5-8UnR-3zk2-wWAb-VP6E-Vij1G5', u'nvme-INTEL_SSDPEDMD400G4_PHFT620400NK400BGN', u'nvme-nvme.8086-504846543632303430304e4b34303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'PHFT620400NK400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.087290", "end": "2019-12-22 22:16:15.383484", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-O32All-jdM5-8UnR-3zk2-wWAb-VP6E-Vij1G5", "nvme-INTEL_SSDPEDMD400G4_PHFT620400NK400BGN", "nvme-nvme.8086-504846543632303430304e4b34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "PHFT620400NK400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:15.296194", "stderr": 2019-12-22T22:16:15.432 INFO:teuthology.task.ansible.out: "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0803619 s, 1.4 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0803619 s, 1.4 GB/s"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:15.447 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccUeNxh5W3AyVvfWk62Sc2jusDXjtphF8kh'], u'uuids': [u'850cf4a6-ebb7-407e-b7c9-04585a3e9e96']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400Q6400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.174041", "end": "2019-12-22 22:16:15.409121", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccUeNxh5W3AyVvfWk62Sc2jusDXjtphF8kh"], "labels": [], "masters": [], "uuids": ["850cf4a6-ebb7-407e-b7c9-04585a3e9e96"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620400Q6400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:15.235080", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.167938 s, 687 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.167938 s, 687 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:15.911 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNIWBINC6y79LorvAaeof2qlHGDe2M56r3'], u'uuids': [u'2617102f-34dd-4a6f-9157-aad2c316a072']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400NK400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.201792", "end": "2019-12-22 22:16:15.876130", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNIWBINC6y79LorvAaeof2qlHGDe2M56r3"], "labels": [], "masters": [], "uuids": ["2617102f-34dd-4a6f-9157-aad2c316a072"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620400NK400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:15.674338", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.195439 s, 590 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.195439 s, 590 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:15.913 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccUyubYCCe66p7fhec857ekxRWepm57mwqE'], u'uuids': [u'dff2aee7-addf-4709-93c8-ac4d39d9a6b1']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400Q6400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.171612", "end": "2019-12-22 22:16:15.888086", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccUyubYCCe66p7fhec857ekxRWepm57mwqE"], "labels": [], "masters": [], "uuids": ["dff2aee7-addf-4709-93c8-ac4d39d9a6b1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "PHFT620400Q6400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:15.716474", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.165846 s, 695 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.165846 s, 695 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:16.407 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccUgKJCcCegdUHAuHCufS36nCU18NB3imhS'], u'uuids': [u'856ebd99-7c67-476c-9a99-7ca0483e9ef0']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400Q6400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.178030", "end": "2019-12-22 22:16:16.371269", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-rohQBxvBdzEjFxh1NWNDNqVDgSwHRccUgKJCcCegdUHAuHCufS36nCU18NB3imhS"], "labels": [], "masters": [], "uuids": ["856ebd99-7c67-476c-9a99-7ca0483e9ef0"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620400Q6400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:16.193239", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.171386 s, 673 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.171386 s, 673 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:16.425 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNXKTPlQpPFE2QYkBrqXGrW4J2A0X3w9SK'], u'uuids': [u'85136b6c-451f-4f69-8d97-3873db09767c']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400NK400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.200397", "end": "2019-12-22 22:16:16.386900", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNXKTPlQpPFE2QYkBrqXGrW4J2A0X3w9SK"], "labels": [], "masters": [], "uuids": ["85136b6c-451f-4f69-8d97-3873db09767c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620400NK400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:16.186503", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.193669 s, 596 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.193669 s, 596 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:16.480 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:16 +0000 (0:00:01.500) 0:04:39.652 ******* 2019-12-22T22:16:16.751 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2GaZGeuvaO6Dz8hEURBUBxwtazA21DG6K'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHMB751300GH480DGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.042848", "end": "2019-12-22 22:16:16.707316", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2GaZGeuvaO6Dz8hEURBUBxwtazA21DG6K"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHMB751300GH480DGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:15.664468", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-22T22:16:16.887 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** changed: [smithi091.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.094272", "end": "2019-12-22 22:16:16.847945", "rc": 0, "start": "2019-12-22 22:16:16.753673", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:16.921 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:16 +0000 (0:00:00.447) 0:04:40.099 ******* 2019-12-22T22:16:16.939 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-22T22:16:16.940 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNQuC9vSdq1miI2lBCNk7WDqxI0SLZwExq'], u'uuids': [u'3af6d14a-1a2a-4a31-8e4a-b8fce9400610']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400NK400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.186619", "end": "2019-12-22 22:16:16.910737", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNQuC9vSdq1miI2lBCNk7WDqxI0SLZwExq"], "labels": [], "masters": [], "uuids": ["3af6d14a-1a2a-4a31-8e4a-b8fce9400610"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620400NK400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:16.724118", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.179872 s, 641 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.179872 s, 641 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:17.315 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2019-12-22T22:16:17.317 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.065438", "end": "2019-12-22 22:16:17.279352", "rc": 0, "start": "2019-12-22 22:16:17.213914", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:17.356 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:17 +0000 (0:00:00.434) 0:04:40.533 ******* 2019-12-22T22:16:17.390 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2019-12-22T22:16:17.433 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:17 +0000 (0:00:00.077) 0:04:40.611 ******* 2019-12-22T22:16:17.451 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-22T22:16:17.453 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNWpc6j19MtDGBPJXOgsRkKWCay0wBfPKY'], u'uuids': [u'774c2207-8848-40f0-89a5-45715203e380']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400NK400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.197070", "end": "2019-12-22 22:16:17.412683", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNWpc6j19MtDGBPJXOgsRkKWCay0wBfPKY"], "labels": [], "masters": [], "uuids": ["774c2207-8848-40f0-89a5-45715203e380"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "PHFT620400NK400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:17.215613", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.190525 s, 605 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.190525 s, 605 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:17.466 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:17.507 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:17 +0000 (0:00:00.073) 0:04:40.685 ******* 2019-12-22T22:16:17.545 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2019-12-22T22:16:17.546 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:17.584 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:17 +0000 (0:00:00.076) 0:04:40.762 ******* 2019-12-22T22:16:17.611 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-22T22:16:17.612 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:17.656 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:17 +0000 (0:00:00.069) 0:04:40.831 ******* 2019-12-22T22:16:17.681 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2019-12-22T22:16:17.683 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:17.723 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:17 +0000 (0:00:00.069) 0:04:40.901 ******* 2019-12-22T22:16:17.936 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-22T22:16:17.940 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNGoy2T1idwrObPPgrWb2bbFCjByE3ImPA'], u'uuids': [u'252c6067-b9f0-4b05-8f26-cb1a1dae4f6f']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT620400NK400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.172820", "end": "2019-12-22 22:16:17.911489", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-9BufiACQYticF2J7w2p2gv6X5Tn49jaNGoy2T1idwrObPPgrWb2bbFCjByE3ImPA"], "labels": [], "masters": [], "uuids": ["252c6067-b9f0-4b05-8f26-cb1a1dae4f6f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT620400NK400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:17.738669", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.166812 s, 691 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.166812 s, 691 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:17.992 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:17 +0000 (0:00:00.269) 0:04:41.170 ******* 2019-12-22T22:16:18.063 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2019-12-22T22:16:18.065 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-22T22:16:18.109 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:18 +0000 (0:00:00.117) 0:04:41.287 ******* 2019-12-22T22:16:18.148 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-22T22:16:18.149 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2bMS8pjkaDY6woKM93C1RpqY4wUjVlotb'], u'uuids': [u'f8f28c17-de7c-4959-88df-522bc6f2aace']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHMB751300GH480DGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.056236", "end": "2019-12-22 22:16:18.100159", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2bMS8pjkaDY6woKM93C1RpqY4wUjVlotb"], "labels": [], "masters": [], "uuids": ["f8f28c17-de7c-4959-88df-522bc6f2aace"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "PHMB751300GH480DGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:17.043923", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-22T22:16:18.151 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ skipping: [smithi091.front.sepia.ceph.com] => (item={u'block_used': 13037562, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227286922, u'size_available': 930967232512, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966956, u'device': u'/dev/sda1', u'inode_used': 88020, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227286922, "block_size": 4096, "block_total": 240324484, "block_used": 13037562, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966956, "inode_total": 61054976, "inode_used": 88020, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930967232512, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:18.190 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:18 +0000 (0:00:00.080) 0:04:41.368 ******* 2019-12-22T22:16:18.217 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2019-12-22T22:16:18.224 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:18.259 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:18 +0000 (0:00:00.064) 0:04:41.433 ******* 2019-12-22T22:16:18.284 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:18.347 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:18 +0000 (0:00:00.092) 0:04:41.525 ******* 2019-12-22T22:16:18.389 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:18.390 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** changed: [smithi107.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.087238", "end": "2019-12-22 22:16:18.356681", "rc": 0, "start": "2019-12-22 22:16:18.269443", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:18.430 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:18 +0000 (0:00:00.081) 0:04:41.607 ******* 2019-12-22T22:16:18.469 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:18 +0000 (0:00:00.040) 0:04:41.647 ******* 2019-12-22T22:16:18.475 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2019-12-22T22:16:18.475 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:18.522 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:18 +0000 (0:00:00.052) 0:04:41.700 ******* 2019-12-22T22:16:18.560 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:18.599 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:18 +0000 (0:00:00.076) 0:04:41.777 ******* 2019-12-22T22:16:18.625 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2019-12-22T22:16:18.627 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:18.668 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:18 +0000 (0:00:00.068) 0:04:41.845 ******* 2019-12-22T22:16:18.866 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2019-12-22T22:16:18.868 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.057981", "end": "2019-12-22 22:16:18.829220", "rc": 0, "start": "2019-12-22 22:16:18.771239", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:18.904 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:18 +0000 (0:00:00.235) 0:04:42.081 ******* 2019-12-22T22:16:18.932 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2019-12-22T22:16:18.975 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:18 +0000 (0:00:00.067) 0:04:42.149 ******* 2019-12-22T22:16:19.009 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2019-12-22T22:16:19.011 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:19.058 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:19 +0000 (0:00:00.078) 0:04:42.227 ******* 2019-12-22T22:16:19.092 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:19.127 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:19 +0000 (0:00:00.078) 0:04:42.305 ******* 2019-12-22T22:16:19.155 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-22T22:16:19.157 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:19.196 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:19 +0000 (0:00:00.068) 0:04:42.374 ******* 2019-12-22T22:16:19.234 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:19.271 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:19 +0000 (0:00:00.075) 0:04:42.449 ******* 2019-12-22T22:16:19.514 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2019-12-22T22:16:19.558 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:19 +0000 (0:00:00.281) 0:04:42.730 ******* 2019-12-22T22:16:19.571 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-22T22:16:19.571 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2MvJGuQjsPwWtyZGdsTaZirk7tY5k2zZr'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHMB751300GH480DGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.088057", "end": "2019-12-22 22:16:19.531725", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2MvJGuQjsPwWtyZGdsTaZirk7tY5k2zZr"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHMB751300GH480DGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:18.443668", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-22T22:16:19.591 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [smithi107.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-22T22:16:19.628 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:19 +0000 (0:00:00.075) 0:04:42.806 ******* 2019-12-22T22:16:19.666 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:19 +0000 (0:00:00.038) 0:04:42.844 ******* 2019-12-22T22:16:19.675 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-22T22:16:19.677 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4HQ2G', u'wwn-0x5000c500871a8b6f'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W4HQ2G', u'wwn': u'0x5000c500871a8b6f', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W4HQ2G-part1', u'wwn-0x5000c500871a8b6f-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQ2G", "wwn-0x5000c500871a8b6f"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4HQ2G-part1", "wwn-0x5000c500871a8b6f-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4HQ2G", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-22T22:16:19.679 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c500871a8b6f"}}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:19.715 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ skipping: [smithi107.front.sepia.ceph.com] => (item={u'block_used': 13037585, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227286899, u'size_available': 930967138304, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966956, u'device': u'/dev/sda1', u'inode_used': 88020, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227286899, "block_size": 4096, "block_total": 240324484, "block_used": 13037585, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966956, "inode_total": 61054976, "inode_used": 88020, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930967138304, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:19.754 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:19 +0000 (0:00:00.088) 0:04:42.932 ******* 2019-12-22T22:16:19.792 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2019-12-22T22:16:19.794 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:19.867 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:19 +0000 (0:00:00.113) 0:04:43.045 ******* 2019-12-22T22:16:19.920 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-22T22:16:19.922 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:19.970 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:19 +0000 (0:00:00.102) 0:04:43.147 ******* 2019-12-22T22:16:20.012 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:20.053 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:20 +0000 (0:00:00.083) 0:04:43.231 ******* 2019-12-22T22:16:20.088 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-0dkcpz-0Fzx-92HN-SU1I-Ptpx-J3VN-djPj3F', u'nvme-INTEL_SSDPED1D480GA_PHMB751300GH480DGN', u'nvme-nvme.8086-50484d42373531333030474834383044474e-494e54454c2053534450454431443438304741-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series', u'support_discard': u'512', u'serial': u'PHMB751300GH480DGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'447.13 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'937703088', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPED1D480GA', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.087171", "end": "2019-12-22 22:16:20.043797", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation Optane SSD 900P Series", "links": {"ids": ["lvm-pv-uuid-0dkcpz-0Fzx-92HN-SU1I-Ptpx-J3VN-djPj3F", "nvme-INTEL_SSDPED1D480GA_PHMB751300GH480DGN", "nvme-nvme.8086-50484d42373531333030474834383044474e-494e54454c2053534450454431443438304741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPED1D480GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "937703088", "sectorsize": "512", "serial": "PHMB751300GH480DGN", "size": "447.13 GB", "support_discard": "512", "vendor": null, "vi 2019-12-22T22:16:20.090 INFO:teuthology.task.ansible.out:rtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:19.956626", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0808177 s, 1.4 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0808177 s, 1.4 GB/s"], "stdout": "", "stdout_lines": []} TASK [testnode : Create volume_groups dict] ************************************ skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:20.131 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:20 +0000 (0:00:00.078) 0:04:43.309 ******* 2019-12-22T22:16:20.159 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2019-12-22T22:16:20.161 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:20.196 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:20 +0000 (0:00:00.065) 0:04:43.374 ******* 2019-12-22T22:16:20.239 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2019-12-22T22:16:20.241 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:20.278 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:20 +0000 (0:00:00.081) 0:04:43.456 ******* 2019-12-22T22:16:20.351 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-22T22:16:20.353 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-22T22:16:20.468 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-22T22:16:20.469 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2hOn7Lm3o9lo3sFoocQs18lTgzPje6X6O'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHMB751300GH480DGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.123835", "end": "2019-12-22 22:16:20.442969", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2hOn7Lm3o9lo3sFoocQs18lTgzPje6X6O"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHMB751300GH480DGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:20.319134", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.118205 s, 976 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.118205 s, 976 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:20.823 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-22T22:16:20.825 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-22T22:16:20.846 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2019-12-22T22:16:20.852 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2019-12-22T22:16:20.885 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:20 +0000 (0:00:00.606) 0:04:44.063 ******* 2019-12-22T22:16:20.954 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-22T22:16:20.957 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2q23HoDfYqzA2b3jG4EziUhKfSIW7SmQJ'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHMB751300GH480DGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.159511", "end": "2019-12-22 22:16:20.919364", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2q23HoDfYqzA2b3jG4EziUhKfSIW7SmQJ"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHMB751300GH480DGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:20.759853", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.152956 s, 754 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.152956 s, 754 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:21.303 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-22T22:16:21.378 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-22T22:16:21.437 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-22T22:16:21.437 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2GaZGeuvaO6Dz8hEURBUBxwtazA21DG6K'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHMB751300GH480DGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.155367", "end": "2019-12-22 22:16:21.402058", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2GaZGeuvaO6Dz8hEURBUBxwtazA21DG6K"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHMB751300GH480DGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:21.246691", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.148704 s, 776 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.148704 s, 776 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:21.767 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-22T22:16:21.769 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-22T22:16:21.844 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-22T22:16:22.279 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-22T22:16:22.336 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:22 +0000 (0:00:01.451) 0:04:45.514 ******* 2019-12-22T22:16:22.344 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-22T22:16:22.672 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2019-12-22T22:16:22.706 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:22 +0000 (0:00:00.370) 0:04:45.884 ******* 2019-12-22T22:16:22.755 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-22T22:16:22.757 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:22.803 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-22T22:16:22.887 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-22T22:16:22.889 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2bMS8pjkaDY6woKM93C1RpqY4wUjVlotb'], u'uuids': [u'f8f28c17-de7c-4959-88df-522bc6f2aace']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHMB751300GH480DGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:01.120419", "end": "2019-12-22 22:16:22.846683", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2bMS8pjkaDY6woKM93C1RpqY4wUjVlotb"], "labels": [], "masters": [], "uuids": ["f8f28c17-de7c-4959-88df-522bc6f2aace"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "PHMB751300GH480DGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:21.726264", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.113026 s, 1.0 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.113026 s, 1.0 GB/s"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:23.050 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-22T22:16:23.050 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-22T22:16:23.317 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-22T22:16:23.321 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-22T22:16:23.388 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:23 +0000 (0:00:00.681) 0:04:46.565 ******* 2019-12-22T22:16:23.394 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-22T22:16:23.396 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-22T22:16:23.398 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-22T22:16:23.400 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2MvJGuQjsPwWtyZGdsTaZirk7tY5k2zZr'], u'uuids': []}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHMB751300GH480DGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.153970", "end": "2019-12-22 22:16:23.333560", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-Py8RGbilcBQqTl8S6JLg0Zeo648VTgd2MvJGuQjsPwWtyZGdsTaZirk7tY5k2zZr"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHMB751300GH480DGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-22 22:16:23.179590", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.147392 s, 783 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.147392 s, 783 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:23.463 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:23 +0000 (0:00:00.075) 0:04:46.641 ******* 2019-12-22T22:16:23.687 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-22T22:16:23.689 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-22T22:16:23.689 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-22T22:16:23.689 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2019-12-22T22:16:23.732 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:23 +0000 (0:00:00.268) 0:04:46.909 ******* 2019-12-22T22:16:23.780 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-22T22:16:23.782 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:23.847 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2019-12-22T22:16:23.849 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.081981", "end": "2019-12-22 22:16:23.813360", "rc": 0, "start": "2019-12-22 22:16:23.731379", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:23.887 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:23 +0000 (0:00:00.155) 0:04:47.065 ******* 2019-12-22T22:16:23.969 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi091.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-22T22:16:24.035 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:24 +0000 (0:00:00.147) 0:04:47.213 ******* 2019-12-22T22:16:24.087 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-22T22:16:24.284 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.050708", "end": "2019-12-22 22:16:24.253774", "rc": 0, "start": "2019-12-22 22:16:24.203066", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:24.350 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:24 +0000 (0:00:00.314) 0:04:47.528 ******* 2019-12-22T22:16:24.388 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-22T22:16:24.431 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:24 +0000 (0:00:00.080) 0:04:47.609 ******* 2019-12-22T22:16:24.441 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2019-12-22T22:16:24.442 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-22T22:16:24.490 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:24 +0000 (0:00:00.059) 0:04:47.668 ******* 2019-12-22T22:16:24.499 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2019-12-22T22:16:24.500 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:24.546 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:24 +0000 (0:00:00.055) 0:04:47.724 ******* 2019-12-22T22:16:24.553 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2019-12-22T22:16:24.563 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2019-12-22T22:16:24.596 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:24 +0000 (0:00:00.049) 0:04:47.774 ******* 2019-12-22T22:16:24.603 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2019-12-22T22:16:24.605 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:24.647 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:24 +0000 (0:00:00.050) 0:04:47.824 ******* 2019-12-22T22:16:24.678 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-22T22:16:24.679 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:24.731 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:24 +0000 (0:00:00.072) 0:04:47.897 ******* 2019-12-22T22:16:24.776 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-22T22:16:24.789 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2019-12-22T22:16:24.791 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:24.832 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:24 +0000 (0:00:00.109) 0:04:48.007 ******* 2019-12-22T22:16:25.092 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-22T22:16:25.094 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-22T22:16:25.175 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:25 +0000 (0:00:00.345) 0:04:48.352 ******* 2019-12-22T22:16:25.181 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2019-12-22T22:16:25.182 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-22T22:16:25.226 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:25 +0000 (0:00:00.051) 0:04:48.404 ******* 2019-12-22T22:16:25.279 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ skipping: [smithi043.front.sepia.ceph.com] => (item={u'block_used': 13037598, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227286886, u'size_available': 930967085056, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966952, u'device': u'/dev/sda1', u'inode_used': 88024, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227286886, "block_size": 4096, "block_total": 240324484, "block_used": 13037598, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966952, "inode_total": 61054976, "inode_used": 88024, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930967085056, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:25.317 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:25 +0000 (0:00:00.091) 0:04:48.495 ******* 2019-12-22T22:16:25.347 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi091.front.sepia.ceph.com] => {"changed": true} 2019-12-22T22:16:25.390 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:25 +0000 (0:00:00.072) 0:04:48.568 ******* 2019-12-22T22:16:25.397 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2019-12-22T22:16:25.399 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:25.440 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:25 +0000 (0:00:00.050) 0:04:48.618 ******* 2019-12-22T22:16:25.490 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-22T22:16:25.492 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:25.531 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:25 +0000 (0:00:00.090) 0:04:48.708 ******* 2019-12-22T22:16:25.538 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-22T22:16:25.578 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:25 +0000 (0:00:00.048) 0:04:48.756 ******* 2019-12-22T22:16:25.584 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-22T22:16:25.585 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:25.632 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:25 +0000 (0:00:00.052) 0:04:48.809 ******* 2019-12-22T22:16:25.649 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** ok: [smithi107.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2019-12-22T22:16:25.702 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:25 +0000 (0:00:00.070) 0:04:48.880 ******* 2019-12-22T22:16:25.707 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2019-12-22T22:16:25.710 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:25.764 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:25 +0000 (0:00:00.061) 0:04:48.941 ******* 2019-12-22T22:16:25.802 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2019-12-22T22:16:25.804 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:25.845 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:25 +0000 (0:00:00.080) 0:04:49.022 ******* 2019-12-22T22:16:25.881 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2019-12-22T22:16:25.883 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:25.918 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:25 +0000 (0:00:00.073) 0:04:49.096 ******* 2019-12-22T22:16:26.144 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2019-12-22T22:16:26.145 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => {"changed": true} 2019-12-22T22:16:26.188 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:26 +0000 (0:00:00.263) 0:04:49.360 ******* 2019-12-22T22:16:26.190 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2019-12-22T22:16:26.193 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2019-12-22T22:16:26.229 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:26 +0000 (0:00:00.046) 0:04:49.407 ******* 2019-12-22T22:16:26.260 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:26.320 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:26 +0000 (0:00:00.090) 0:04:49.498 ******* 2019-12-22T22:16:26.357 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:26.404 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:26 +0000 (0:00:00.084) 0:04:49.582 ******* 2019-12-22T22:16:26.432 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2019-12-22T22:16:26.433 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:26.473 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:26 +0000 (0:00:00.068) 0:04:49.650 ******* 2019-12-22T22:16:26.507 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2019-12-22T22:16:26.508 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2019-12-22T22:16:26.509 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:26.549 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:26 +0000 (0:00:00.076) 0:04:49.726 ******* 2019-12-22T22:16:26.584 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:26 +0000 (0:00:00.035) 0:04:49.762 ******* 2019-12-22T22:16:26.613 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:26.652 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:26 +0000 (0:00:00.067) 0:04:49.830 ******* 2019-12-22T22:16:26.679 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2019-12-22T22:16:26.689 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:26.720 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:26 +0000 (0:00:00.065) 0:04:49.896 ******* 2019-12-22T22:16:26.745 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:26.792 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:26 +0000 (0:00:00.074) 0:04:49.970 ******* 2019-12-22T22:16:27.071 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-22T22:16:27.074 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-22T22:16:27.308 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2019-12-22T22:16:27.310 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2019-12-22T22:16:27.353 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:27 +0000 (0:00:00.557) 0:04:50.527 ******* 2019-12-22T22:16:27.412 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:27.445 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:27 +0000 (0:00:00.095) 0:04:50.623 ******* 2019-12-22T22:16:27.491 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2019-12-22T22:16:27.493 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:27.543 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:27 +0000 (0:00:00.096) 0:04:50.720 ******* 2019-12-22T22:16:27.571 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-22T22:16:27.660 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2019-12-22T22:16:27.708 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:27 +0000 (0:00:00.162) 0:04:50.883 ******* 2019-12-22T22:16:27.741 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:27.776 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:27 +0000 (0:00:00.070) 0:04:50.954 ******* 2019-12-22T22:16:27.835 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2019-12-22T22:16:27.837 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:27.874 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:27 +0000 (0:00:00.098) 0:04:51.052 ******* 2019-12-22T22:16:27.905 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2019-12-22T22:16:27.908 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:27.946 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:27 +0000 (0:00:00.071) 0:04:51.123 ******* 2019-12-22T22:16:27.974 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2019-12-22T22:16:27.976 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:28.012 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:28 +0000 (0:00:00.065) 0:04:51.189 ******* 2019-12-22T22:16:28.051 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2019-12-22T22:16:28.053 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:28.090 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:28 +0000 (0:00:00.078) 0:04:51.267 ******* 2019-12-22T22:16:28.095 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-22T22:16:28.097 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-22T22:16:28.122 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:28.157 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:28 +0000 (0:00:00.067) 0:04:51.335 ******* 2019-12-22T22:16:28.184 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2019-12-22T22:16:28.186 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:28.223 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:28 +0000 (0:00:00.066) 0:04:51.401 ******* 2019-12-22T22:16:28.464 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2019-12-22T22:16:28.530 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:28 +0000 (0:00:00.306) 0:04:51.708 ******* 2019-12-22T22:16:28.550 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-22T22:16:28.572 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:28.611 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:28 +0000 (0:00:00.081) 0:04:51.789 ******* 2019-12-22T22:16:28.732 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2019-12-22T22:16:28.737 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2019-12-22T22:16:28.788 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:28 +0000 (0:00:00.175) 0:04:51.965 ******* 2019-12-22T22:16:28.816 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2019-12-22T22:16:28.818 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:28.870 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:28 +0000 (0:00:00.079) 0:04:52.044 ******* 2019-12-22T22:16:28.895 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:28.934 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:28 +0000 (0:00:00.066) 0:04:52.111 ******* 2019-12-22T22:16:29.028 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-22T22:16:29.030 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-22T22:16:29.077 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:29 +0000 (0:00:00.138) 0:04:52.250 ******* 2019-12-22T22:16:29.153 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "sntp.service ntpdate.service -.mount syslog.target tmp.mount systemd-journald.socket basic.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "multi-user.target shutdown.target chronyd.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": " 2019-12-22T22:16:29.156 INFO:teuthology.task.ansible.out:0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "-.mount basic.target", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551 2019-12-22T22:16:29.158 INFO:teuthology.task.ansible.out:615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-22T22:16:29.199 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:29 +0000 (0:00:00.127) 0:04:52.377 ******* 2019-12-22T22:16:29.413 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2019-12-22T22:16:29.454 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:29 +0000 (0:00:00.252) 0:04:52.630 ******* 2019-12-22T22:16:29.503 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-22T22:16:29.505 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:29.827 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-22T22:16:29.946 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2019-12-22T22:16:29.947 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2019-12-22T22:16:30.011 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:30 +0000 (0:00:00.558) 0:04:53.188 ******* 2019-12-22T22:16:30.054 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2019-12-22T22:16:30.055 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:30.112 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:30 +0000 (0:00:00.100) 0:04:53.289 ******* 2019-12-22T22:16:30.147 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-22T22:16:30.149 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-22T22:16:30.191 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2019-12-22T22:16:30.193 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2019-12-22T22:16:30.280 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:30 +0000 (0:00:00.168) 0:04:53.458 ******* 2019-12-22T22:16:30.311 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:30.352 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:30 +0000 (0:00:00.069) 0:04:53.527 ******* 2019-12-22T22:16:30.382 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:30.431 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:30 +0000 (0:00:00.082) 0:04:53.609 ******* 2019-12-22T22:16:30.483 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-22T22:16:30.486 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-22T22:16:30.618 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2019-12-22T22:16:30.620 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "ntpdate.service tmp.mount systemd-journald.socket system.slice basic.target sntp.service syslog.target -.mount", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "multi-user.target shutdown.target chronyd.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target 2019-12-22T22:16:30.621 INFO:teuthology.task.ansible.out:", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "1844674407370955161 2019-12-22T22:16:30.622 INFO:teuthology.task.ansible.out:5", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127510", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127510", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "-.mount basic.target", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec 2019-12-22T22:16:30.624 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-22T22:16:30.688 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:30 +0000 (0:00:00.254) 0:04:53.863 ******* 2019-12-22T22:16:30.767 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2019-12-22T22:16:30.816 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:30 +0000 (0:00:00.130) 0:04:53.994 ******* 2019-12-22T22:16:30.828 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** changed: [smithi043.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-22T22:16:30.878 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:30 +0000 (0:00:00.054) 0:04:54.049 ******* 2019-12-22T22:16:31.250 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2019-12-22T22:16:31.252 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-22T22:16:31.295 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:31 +0000 (0:00:00.421) 0:04:54.470 ******* 2019-12-22T22:16:31.332 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** ok: [smithi043.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2019-12-22T22:16:31.391 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:31 +0000 (0:00:00.099) 0:04:54.569 ******* 2019-12-22T22:16:31.635 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2019-12-22T22:16:31.636 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2019-12-22T22:16:31.680 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:31 +0000 (0:00:00.288) 0:04:54.858 ******* 2019-12-22T22:16:31.709 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2019-12-22T22:16:31.711 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:31.752 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:31 +0000 (0:00:00.071) 0:04:54.930 ******* 2019-12-22T22:16:31.776 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2019-12-22T22:16:31.813 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:31 +0000 (0:00:00.060) 0:04:54.990 ******* 2019-12-22T22:16:31.822 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi043.front.sepia.ceph.com] => {"changed": true} TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:31.886 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:31 +0000 (0:00:00.073) 0:04:55.064 ******* 2019-12-22T22:16:31.976 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:31 +0000 (0:00:00.088) 0:04:55.152 ******* 2019-12-22T22:16:32.338 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2019-12-22T22:16:32.339 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2019-12-22T22:16:32.403 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:32 +0000 (0:00:00.428) 0:04:55.581 ******* 2019-12-22T22:16:32.413 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2019-12-22T22:16:32.415 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2019-12-22T22:16:32.455 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:32 +0000 (0:00:00.051) 0:04:55.632 ******* 2019-12-22T22:16:32.498 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:32.570 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:32 +0000 (0:00:00.115) 0:04:55.747 ******* 2019-12-22T22:16:32.617 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2019-12-22T22:16:32.618 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:32.679 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:32 +0000 (0:00:00.107) 0:04:55.854 ******* 2019-12-22T22:16:32.719 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:32.776 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:32 +0000 (0:00:00.099) 0:04:55.954 ******* 2019-12-22T22:16:32.833 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:32.872 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:32 +0000 (0:00:00.095) 0:04:56.050 ******* 2019-12-22T22:16:33.052 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:33.106 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:33 +0000 (0:00:00.234) 0:04:56.284 ******* 2019-12-22T22:16:33.403 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2019-12-22T22:16:33.404 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:33.407 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2019-12-22T22:16:33.449 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:33 +0000 (0:00:00.342) 0:04:56.627 ******* 2019-12-22T22:16:33.487 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:33 +0000 (0:00:00.036) 0:04:56.663 ******* 2019-12-22T22:16:33.492 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:33.534 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:33 +0000 (0:00:00.048) 0:04:56.711 ******* 2019-12-22T22:16:33.786 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip2", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 19.3.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 19.3.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2019-12-22T22:16:33.836 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:33 +0000 (0:00:00.302) 0:04:57.013 ******* 2019-12-22T22:16:33.840 INFO:teuthology.task.ansible.out: 2019-12-22T22:16:33.868 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2019-12-22T22:16:33.870 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:33.909 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:33 +0000 (0:00:00.073) 0:04:57.087 ******* 2019-12-22T22:16:33.960 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2019-12-22T22:16:33.989 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:33 +0000 (0:00:00.078) 0:04:57.166 ******* 2019-12-22T22:16:34.049 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2019-12-22T22:16:34.050 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2019-12-22T22:16:34.103 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:34 +0000 (0:00:00.113) 0:04:57.280 ******* 2019-12-22T22:16:34.145 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:34.187 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:34 +0000 (0:00:00.081) 0:04:57.361 ******* 2019-12-22T22:16:34.214 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:34.252 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:34 +0000 (0:00:00.066) 0:04:57.428 ******* 2019-12-22T22:16:34.293 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2019-12-22T22:16:34.297 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2019-12-22T22:16:34.349 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:34 +0000 (0:00:00.099) 0:04:57.527 ******* 2019-12-22T22:16:34.391 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2019-12-22T22:16:34.393 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:34.452 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:34 +0000 (0:00:00.102) 0:04:57.630 ******* 2019-12-22T22:16:34.495 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** ok: [smithi091.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2019-12-22T22:16:34.540 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:34 +0000 (0:00:00.087) 0:04:57.718 ******* 2019-12-22T22:16:34.589 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:34.627 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:34 +0000 (0:00:00.087) 0:04:57.805 ******* 2019-12-22T22:16:34.957 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2019-12-22T22:16:34.959 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2019-12-22T22:16:35.007 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:35 +0000 (0:00:00.379) 0:04:58.184 ******* 2019-12-22T22:16:35.021 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip2", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 19.3.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 19.3.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2019-12-22T22:16:35.068 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:35 +0000 (0:00:00.061) 0:04:58.246 ******* 2019-12-22T22:16:35.072 INFO:teuthology.task.ansible.out: 2019-12-22T22:16:35.109 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2019-12-22T22:16:35.110 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:35.154 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:35 +0000 (0:00:00.081) 0:04:58.327 ******* 2019-12-22T22:16:35.187 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2019-12-22T22:16:35.189 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2019-12-22T22:16:35.225 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:35 +0000 (0:00:00.075) 0:04:58.403 ******* 2019-12-22T22:16:35.276 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2019-12-22T22:16:35.328 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:35 +0000 (0:00:00.102) 0:04:58.506 ******* 2019-12-22T22:16:35.335 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2019-12-22T22:16:35.335 INFO:teuthology.task.ansible.out:changed: [smithi091.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2019-12-22T22:16:35.381 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:35 +0000 (0:00:00.052) 0:04:58.559 ******* 2019-12-22T22:16:35.393 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:35.437 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:35 +0000 (0:00:00.055) 0:04:58.614 ******* 2019-12-22T22:16:35.445 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2019-12-22T22:16:35.446 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:35.496 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:35 +0000 (0:00:00.058) 0:04:58.673 ******* 2019-12-22T22:16:35.535 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2019-12-22T22:16:35.537 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} TASK [testnode : Check for presence of /etc/network/interfaces] **************** ok: [smithi107.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2019-12-22T22:16:35.596 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:35 +0000 (0:00:00.095) 0:04:58.769 ******* 2019-12-22T22:16:35.601 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [smithi091.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi091.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:35.601 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:35.656 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:35 +0000 (0:00:00.063) 0:04:58.832 ******* 2019-12-22T22:16:35.662 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:35.712 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:35 +0000 (0:00:00.051) 0:04:58.884 ******* 2019-12-22T22:16:35.734 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2019-12-22T22:16:35.735 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:35.760 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2019-12-22T22:16:35.762 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2019-12-22T22:16:35.765 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2019-12-22T22:16:35.767 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:35.809 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:35 +0000 (0:00:00.103) 0:04:58.987 ******* 2019-12-22T22:16:35.817 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:35.834 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:35.835 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:35.903 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:35 +0000 (0:00:00.093) 0:04:59.081 ******* 2019-12-22T22:16:35.915 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:35.959 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:35 +0000 (0:00:00.055) 0:04:59.137 ******* 2019-12-22T22:16:35.969 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:35.972 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "syslog.target basic.target sntp.service -.mount ntpdate.service tmp.mount systemd-journald.socket system.slice", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target multi-user.target chronyd.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": " 2019-12-22T22:16:35.973 INFO:teuthology.task.ansible.out:0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127535", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127535", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551 2019-12-22T22:16:35.975 INFO:teuthology.task.ansible.out:615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-22T22:16:36.014 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:36 +0000 (0:00:00.054) 0:04:59.191 ******* 2019-12-22T22:16:36.064 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:36 +0000 (0:00:00.050) 0:04:59.242 ******* 2019-12-22T22:16:36.113 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:36.149 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:36 +0000 (0:00:00.082) 0:04:59.324 ******* 2019-12-22T22:16:36.185 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2019-12-22T22:16:36.187 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:36.237 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:36 +0000 (0:00:00.090) 0:04:59.415 ******* 2019-12-22T22:16:36.266 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2019-12-22T22:16:36.267 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:36.269 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** changed: [smithi107.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2019-12-22T22:16:36.309 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:36 +0000 (0:00:00.071) 0:04:59.487 ******* 2019-12-22T22:16:36.357 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:36 +0000 (0:00:00.042) 0:04:59.529 ******* 2019-12-22T22:16:36.362 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:36.417 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:36 +0000 (0:00:00.065) 0:04:59.594 ******* 2019-12-22T22:16:36.446 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2019-12-22T22:16:36.448 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:36.501 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:36 +0000 (0:00:00.084) 0:04:59.679 ******* 2019-12-22T22:16:36.547 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2019-12-22T22:16:36.550 INFO:teuthology.task.ansible.out:skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:36.610 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:36 +0000 (0:00:00.109) 0:04:59.788 ******* 2019-12-22T22:16:36.660 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi091.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:36.722 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:36 +0000 (0:00:00.112) 0:04:59.900 ******* 2019-12-22T22:16:36.734 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2019-12-22T22:16:36.735 INFO:teuthology.task.ansible.out:changed: [smithi107.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2019-12-22T22:16:36.808 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:36 +0000 (0:00:00.086) 0:04:59.986 ******* 2019-12-22T22:16:36.851 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2019-12-22T22:16:36.853 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:36.890 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:36 +0000 (0:00:00.081) 0:05:00.068 ******* 2019-12-22T22:16:36.942 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [smithi107.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:36.954 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:36.959 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi107.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:36.972 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [smithi107.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:37.037 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:37 +0000 (0:00:00.139) 0:05:00.208 ******* 2019-12-22T22:16:37.105 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:37 +0000 (0:00:00.072) 0:05:00.280 ******* 2019-12-22T22:16:37.116 INFO:teuthology.task.ansible.out: 2019-12-22T22:16:37.118 INFO:teuthology.task.ansible.out:TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi091.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.017217", "end": "2019-12-22 22:16:37.030785", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2019-12-22 22:16:37.013568", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:37.120 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:37.198 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:37 +0000 (0:00:00.093) 0:05:00.374 ******* 2019-12-22T22:16:37.208 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** skipping: [smithi107.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:37.222 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi107.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:37.240 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:37.245 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Define net_to_configure var] ********************************** skipping: [smithi107.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:37.289 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:37 +0000 (0:00:00.092) 0:05:00.467 ******* 2019-12-22T22:16:37.337 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:37 +0000 (0:00:00.041) 0:05:00.509 ******* 2019-12-22T22:16:37.371 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:37.419 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:37 +0000 (0:00:00.087) 0:05:00.596 ******* 2019-12-22T22:16:37.426 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-22T22:16:37.428 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.005821", "end": "2019-12-22 22:16:37.361145", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2019-12-22 22:16:37.355324", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:37.450 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2019-12-22T22:16:37.452 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:37.520 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:37 +0000 (0:00:00.100) 0:05:00.697 ******* 2019-12-22T22:16:37.606 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:37.656 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:37 +0000 (0:00:00.136) 0:05:00.833 ******* 2019-12-22T22:16:37.663 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2019-12-22T22:16:37.715 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:37 +0000 (0:00:00.058) 0:05:00.892 ******* 2019-12-22T22:16:37.724 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2019-12-22T22:16:37.726 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:37.825 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:37 +0000 (0:00:00.110) 0:05:01.003 ******* 2019-12-22T22:16:37.857 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-22T22:16:37.860 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.007212", "end": "2019-12-22 22:16:37.705587", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2019-12-22 22:16:37.698375", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:37.877 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:37.924 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:37 +0000 (0:00:00.099) 0:05:01.102 ******* 2019-12-22T22:16:37.987 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:38.036 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:38 +0000 (0:00:00.111) 0:05:01.214 ******* 2019-12-22T22:16:38.075 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2019-12-22T22:16:38.077 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:38.129 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:38 +0000 (0:00:00.093) 0:05:01.307 ******* 2019-12-22T22:16:38.136 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-22T22:16:38.137 INFO:teuthology.task.ansible.out:ok: [smithi091.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.009043", "end": "2019-12-22 22:16:38.072694", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2019-12-22 22:16:38.063651", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:38.205 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:38 +0000 (0:00:00.075) 0:05:01.383 ******* 2019-12-22T22:16:38.212 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2019-12-22T22:16:38.213 INFO:teuthology.task.ansible.out:skipping: [smithi107.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:38.260 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:38 +0000 (0:00:00.054) 0:05:01.437 ******* 2019-12-22T22:16:38.267 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2019-12-22T22:16:38.319 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:38 +0000 (0:00:00.055) 0:05:01.493 ******* 2019-12-22T22:16:38.656 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** changed: [smithi091.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-22T22:16:38.806 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2019-12-22T22:16:38.841 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:38 +0000 (0:00:00.526) 0:05:02.019 ******* 2019-12-22T22:16:39.626 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-22T22:16:39.627 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:01.017348", "end": "2019-12-22 22:16:39.581744", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2019-12-22 22:16:38.564396", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:39.953 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.007832", "end": "2019-12-22 22:16:39.912183", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2019-12-22 22:16:39.904351", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:40.261 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.006607", "end": "2019-12-22 22:16:40.226650", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2019-12-22 22:16:40.220043", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:40.369 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip2", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 19.3.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 19.3.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2019-12-22T22:16:40.429 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:40 +0000 (0:00:01.587) 0:05:03.606 ******* 2019-12-22T22:16:40.465 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:40.497 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:40 +0000 (0:00:00.065) 0:05:03.672 ******* 2019-12-22T22:16:40.533 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2019-12-22T22:16:40.534 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2019-12-22T22:16:40.583 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:40 +0000 (0:00:00.088) 0:05:03.760 ******* 2019-12-22T22:16:40.588 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-22T22:16:40.591 INFO:teuthology.task.ansible.out:ok: [smithi107.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.008612", "end": "2019-12-22 22:16:40.538469", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2019-12-22 22:16:40.529857", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:40.635 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:40 +0000 (0:00:00.051) 0:05:03.812 ******* 2019-12-22T22:16:40.671 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2019-12-22T22:16:40.709 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:40 +0000 (0:00:00.074) 0:05:03.887 ******* 2019-12-22T22:16:40.895 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2019-12-22T22:16:40.897 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2019-12-22T22:16:40.956 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:40 +0000 (0:00:00.247) 0:05:04.134 ******* 2019-12-22T22:16:40.958 INFO:teuthology.task.ansible.out: 2019-12-22T22:16:40.995 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2019-12-22T22:16:40.997 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:41.040 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:41 +0000 (0:00:00.084) 0:05:04.218 ******* 2019-12-22T22:16:41.053 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** changed: [smithi107.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-22T22:16:41.094 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2019-12-22T22:16:41.096 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2019-12-22T22:16:41.135 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:41 +0000 (0:00:00.094) 0:05:04.313 ******* 2019-12-22T22:16:41.191 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:41.229 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:41 +0000 (0:00:00.094) 0:05:04.407 ******* 2019-12-22T22:16:41.559 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2019-12-22T22:16:41.561 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2019-12-22T22:16:41.611 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:41 +0000 (0:00:00.381) 0:05:04.789 ******* 2019-12-22T22:16:41.941 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2019-12-22T22:16:41.943 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2019-12-22T22:16:41.993 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:41 +0000 (0:00:00.382) 0:05:05.171 ******* 2019-12-22T22:16:42.021 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:42.065 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:42 +0000 (0:00:00.072) 0:05:05.243 ******* 2019-12-22T22:16:42.070 INFO:teuthology.task.ansible.out: 2019-12-22T22:16:42.114 INFO:teuthology.task.ansible.out: 2019-12-22T22:16:42.116 INFO:teuthology.task.ansible.out:TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [smithi043.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:42.118 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:42.120 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:42.131 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:42.132 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} 2019-12-22T22:16:42.202 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:42 +0000 (0:00:00.136) 0:05:05.380 ******* 2019-12-22T22:16:42.241 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2019-12-22T22:16:42.244 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'ens1f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:42.261 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:42.267 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:42.284 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi043.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'ens1f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:42.348 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:42 +0000 (0:00:00.125) 0:05:05.506 ******* 2019-12-22T22:16:42.356 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-22T22:16:42.370 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:42.401 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:42 +0000 (0:00:00.072) 0:05:05.578 ******* 2019-12-22T22:16:42.428 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2019-12-22T22:16:42.430 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:42.489 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:42 +0000 (0:00:00.087) 0:05:05.666 ******* 2019-12-22T22:16:42.517 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2019-12-22T22:16:42.519 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:42.558 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:42 +0000 (0:00:00.069) 0:05:05.736 ******* 2019-12-22T22:16:42.587 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2019-12-22T22:16:42.588 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:42.627 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:42 +0000 (0:00:00.068) 0:05:05.804 ******* 2019-12-22T22:16:42.665 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2019-12-22T22:16:42.667 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:42.714 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:42 +0000 (0:00:00.087) 0:05:05.892 ******* 2019-12-22T22:16:42.745 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2019-12-22T22:16:42.747 INFO:teuthology.task.ansible.out:skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:42.812 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:42 +0000 (0:00:00.097) 0:05:05.990 ******* 2019-12-22T22:16:42.861 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:42.899 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:42 +0000 (0:00:00.087) 0:05:06.077 ******* 2019-12-22T22:16:42.942 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi043.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-22T22:16:42.985 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:42 +0000 (0:00:00.085) 0:05:06.163 ******* 2019-12-22T22:16:43.362 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-22T22:16:43.363 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.013464", "end": "2019-12-22 22:16:43.310299", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2019-12-22 22:16:43.296835", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:43.711 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.008335", "end": "2019-12-22 22:16:43.662836", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2019-12-22 22:16:43.654501", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:44.052 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.006757", "end": "2019-12-22 22:16:44.004211", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2019-12-22 22:16:43.997454", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:44.379 INFO:teuthology.task.ansible.out:ok: [smithi043.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.007829", "end": "2019-12-22 22:16:44.340627", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2019-12-22 22:16:44.332798", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-22T22:16:44.421 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:44 +0000 (0:00:01.435) 0:05:07.599 ******* 2019-12-22T22:16:44.448 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2019-12-22T22:16:44.500 INFO:teuthology.task.ansible.out:Sunday 22 December 2019 22:16:44 +0000 (0:00:00.075) 0:05:07.675 ******* 2019-12-22T22:16:44.831 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2019-12-22T22:16:44.832 INFO:teuthology.task.ansible.out:changed: [smithi043.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-22T22:16:44.960 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2019-12-22T22:16:44.964 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** 2019-12-22T22:16:44.966 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2019-12-22T22:16:44.967 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched PLAY RECAP ********************************************************************* smithi043.front.sepia.ceph.com : ok=121 changed=41 unreachable=0 failed=0 skipped=161 rescued=0 ignored=3 smithi091.front.sepia.ceph.com : ok=121 changed=41 unreachable=0 failed=0 skipped=161 rescued=0 ignored=3 smithi107.front.sepia.ceph.com : ok=121 changed=41 unreachable=0 failed=0 skipped=161 rescued=0 ignored=3 Sunday 22 December 2019 22:16:44 +0000 (0:00:00.469) 0:05:08.144 ******* =============================================================================== 2019-12-22T22:16:44.969 INFO:teuthology.task.ansible.out:users : Create all admin users with sudo access. ----------------------- 37.17s users : Update authorized_keys using the keys repo --------------------- 35.07s common : nrpe - Ensure SELinux policy is up to date -------------------- 24.08s testnode : Install packages -------------------------------------------- 17.55s users : Remove revoked users ------------------------------------------- 14.57s common : Update yum cache ---------------------------------------------- 10.25s common : nrpe - Remove SELinux policy package --------------------------- 4.28s testnode : Ensure ceph dependency packages are not present. ------------- 3.54s common : nrpe - Load SELinux policy package ----------------------------- 3.46s Gathering Facts --------------------------------------------------------- 3.40s testnode : Zap all non-root disks --------------------------------------- 3.22s testnode : Ensure ceph packages are not present. ------------------------ 3.09s ansible-managed : Add authorized keys for the ansible user. ------------- 2.89s testnode : Install packages via pip ------------------------------------- 1.59s ansible-managed : Create the cephlab_sudo sudoers.d file. --------------- 1.46s testnode : List any leftover Ceph artifacts from previous jobs ---------- 1.44s Gathering Facts --------------------------------------------------------- 1.03s common : Install nrpe package and dependencies (RHEL/CentOS) ------------ 1.00s common : Add krb5 config file ------------------------------------------- 0.98s common : nrpe - Copy SELinux type enforcement file ---------------------- 0.94s 2019-12-22T22:16:45.274 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi091.front.sepia.ceph.com'), Remote(name='ubuntu@smithi043.front.sepia.ceph.com'), Remote(name='ubuntu@smithi107.front.sepia.ceph.com')] 2019-12-22T22:16:45.276 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-22T22:16:45.277 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi091.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:16:45.383 INFO:teuthology.orchestra.run.smithi091:> true 2019-12-22T22:16:45.771 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-22T22:16:45.772 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi043.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:16:45.877 INFO:teuthology.orchestra.run.smithi043:> true 2019-12-22T22:16:46.268 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-22T22:16:46.269 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi107.front.sepia.ceph.com', 'timeout': 60} 2019-12-22T22:16:46.391 INFO:teuthology.orchestra.run.smithi107:> true 2019-12-22T22:16:46.755 INFO:teuthology.run_tasks:Running task clock... 2019-12-22T22:16:46.764 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2019-12-22T22:16:46.765 INFO:teuthology.orchestra.run.smithi091:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2019-12-22T22:16:46.900 INFO:teuthology.orchestra.run.smithi091.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2019-12-22T22:16:56.987 INFO:teuthology.orchestra.run.smithi091.stdout:ntpd: time slew -0.000510s 2019-12-22T22:16:57.028 INFO:teuthology.orchestra.run.smithi091.stderr:Failed to start ntp.service: Unit not found. 2019-12-22T22:16:57.118 INFO:teuthology.orchestra.run.smithi091.stdout: remote refid st t when poll reach delay offset jitter 2019-12-22T22:16:57.172 INFO:teuthology.orchestra.run.smithi091.stdout:============================================================================== 2019-12-22T22:16:57.172 INFO:teuthology.orchestra.run.smithi091.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-22T22:16:57.172 INFO:teuthology.orchestra.run.smithi091.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-22T22:16:57.172 INFO:teuthology.orchestra.run.smithi091.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-22T22:16:57.172 INFO:teuthology.orchestra.run.smithi091.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-22T22:16:57.174 INFO:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2019-12-22T22:16:57.322 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2019-12-22T22:17:07.424 INFO:teuthology.orchestra.run.smithi043.stdout:ntpd: time slew -0.000299s 2019-12-22T22:17:07.465 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntp.service: Unit not found. 2019-12-22T22:17:07.549 INFO:teuthology.orchestra.run.smithi043.stdout: remote refid st t when poll reach delay offset jitter 2019-12-22T22:17:07.562 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================== 2019-12-22T22:17:07.563 INFO:teuthology.orchestra.run.smithi043.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-22T22:17:07.563 INFO:teuthology.orchestra.run.smithi043.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-22T22:17:07.563 INFO:teuthology.orchestra.run.smithi043.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-22T22:17:07.563 INFO:teuthology.orchestra.run.smithi043.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-22T22:17:07.564 INFO:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2019-12-22T22:17:07.703 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2019-12-22T22:17:17.794 INFO:teuthology.orchestra.run.smithi107.stdout:ntpd: time slew -0.001566s 2019-12-22T22:17:17.835 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit not found. 2019-12-22T22:17:17.916 INFO:teuthology.orchestra.run.smithi107.stdout: remote refid st t when poll reach delay offset jitter 2019-12-22T22:17:17.917 INFO:teuthology.orchestra.run.smithi107.stdout:============================================================================== 2019-12-22T22:17:17.917 INFO:teuthology.orchestra.run.smithi107.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-22T22:17:17.917 INFO:teuthology.orchestra.run.smithi107.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-22T22:17:17.917 INFO:teuthology.orchestra.run.smithi107.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-22T22:17:17.917 INFO:teuthology.orchestra.run.smithi107.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-22T22:17:17.918 INFO:teuthology.run_tasks:Running task install... 2019-12-22T22:17:17.931 DEBUG:teuthology.task.install:project ceph 2019-12-22T22:17:17.931 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': '99b509ffd4dd354919b08b38ba83b998fbfdc107'}} 2019-12-22T22:17:17.931 DEBUG:teuthology.task.install:config {'sha1': '99b509ffd4dd354919b08b38ba83b998fbfdc107', 'extra_packages': ['librados2', 'python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'branch': 'luminous', 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-diskprediction-cloud', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-immutable-object-cache', 'ceph-base', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'librados-devel']} 2019-12-22T22:17:17.931 INFO:teuthology.task.install:Using flavor: basic 2019-12-22T22:17:17.947 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-mgr', 'ceph-fuse', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2019-12-22T22:17:17.947 INFO:teuthology.task.install:extra packages: ['librados2', 'python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'] 2019-12-22T22:17:17.947 INFO:teuthology.orchestra.run.smithi091:> sudo lsb_release -is 2019-12-22T22:17:18.115 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS 2019-12-22T22:17:18.117 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-22T22:17:18.118 INFO:teuthology.orchestra.run.smithi043:> sudo lsb_release -is 2019-12-22T22:17:18.118 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'sha1': '99b509ffd4dd354919b08b38ba83b998fbfdc107', 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-diskprediction-cloud', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-immutable-object-cache', 'ceph-base', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'librados-devel'], 'debuginfo': None, 'install_ceph_packages': True, 'wait_for_package': False, 'downgrade_packages': [], 'project': 'ceph', 'tag': None, 'extra_packages': ['librados2', 'python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'branch': 'luminous', 'packages': {}, 'repos_only': False, 'extra_system_packages': [], 'flavor': 'basic', 'extras': None} 2019-12-22T22:17:18.118 INFO:teuthology.orchestra.run.smithi091:> sudo lsb_release -is 2019-12-22T22:17:18.236 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS 2019-12-22T22:17:18.238 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-22T22:17:18.238 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2019-12-22T22:17:18.238 INFO:teuthology.packaging:ref: None 2019-12-22T22:17:18.239 INFO:teuthology.packaging:tag: None 2019-12-22T22:17:18.239 INFO:teuthology.packaging:branch: luminous 2019-12-22T22:17:18.239 INFO:teuthology.packaging:sha1: 99b509ffd4dd354919b08b38ba83b998fbfdc107 2019-12-22T22:17:18.239 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F7%2Fx86_64&ref=luminous 2019-12-22T22:17:18.293 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS 2019-12-22T22:17:18.294 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-22T22:17:18.294 INFO:teuthology.orchestra.run.smithi107:> sudo lsb_release -is 2019-12-22T22:17:18.294 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'sha1': '99b509ffd4dd354919b08b38ba83b998fbfdc107', 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-diskprediction-cloud', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-immutable-object-cache', 'ceph-base', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'librados-devel'], 'debuginfo': None, 'install_ceph_packages': True, 'wait_for_package': False, 'downgrade_packages': [], 'project': 'ceph', 'tag': None, 'extra_packages': ['librados2', 'python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'branch': 'luminous', 'packages': {}, 'repos_only': False, 'extra_system_packages': [], 'flavor': 'basic', 'extras': None} 2019-12-22T22:17:18.294 INFO:teuthology.orchestra.run.smithi043:> sudo lsb_release -is 2019-12-22T22:17:18.442 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS 2019-12-22T22:17:18.444 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-22T22:17:18.444 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2019-12-22T22:17:18.445 INFO:teuthology.packaging:ref: None 2019-12-22T22:17:18.445 INFO:teuthology.packaging:tag: None 2019-12-22T22:17:18.445 INFO:teuthology.packaging:branch: luminous 2019-12-22T22:17:18.445 INFO:teuthology.packaging:sha1: 99b509ffd4dd354919b08b38ba83b998fbfdc107 2019-12-22T22:17:18.445 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F7%2Fx86_64&ref=luminous 2019-12-22T22:17:18.457 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS 2019-12-22T22:17:18.460 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-22T22:17:18.460 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'sha1': '99b509ffd4dd354919b08b38ba83b998fbfdc107', 'exclude_packages': ['librados3', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-diskprediction-cloud', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'cephadm', 'ceph-immutable-object-cache', 'ceph-base', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'librados-devel'], 'debuginfo': None, 'install_ceph_packages': True, 'wait_for_package': False, 'downgrade_packages': [], 'project': 'ceph', 'tag': None, 'extra_packages': ['librados2', 'python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'branch': 'luminous', 'packages': {}, 'repos_only': False, 'extra_system_packages': [], 'flavor': 'basic', 'extras': None} 2019-12-22T22:17:18.460 INFO:teuthology.orchestra.run.smithi107:> sudo lsb_release -is 2019-12-22T22:17:18.580 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS 2019-12-22T22:17:18.582 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-22T22:17:18.582 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2019-12-22T22:17:18.582 INFO:teuthology.packaging:ref: None 2019-12-22T22:17:18.582 INFO:teuthology.packaging:tag: None 2019-12-22T22:17:18.582 INFO:teuthology.packaging:branch: luminous 2019-12-22T22:17:18.583 INFO:teuthology.packaging:sha1: 99b509ffd4dd354919b08b38ba83b998fbfdc107 2019-12-22T22:17:18.583 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F7%2Fx86_64&ref=luminous 2019-12-22T22:17:18.705 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/luminous/a6ab1827d7a5dab1ee7562a6de0ae213598c4a94/centos/7/flavors/default/ 2019-12-22T22:17:18.706 INFO:teuthology.task.install.rpm:Package version is 12.2.12-835.ga6ab182 2019-12-22T22:17:18.766 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/luminous/a6ab1827d7a5dab1ee7562a6de0ae213598c4a94/centos/7/flavors/default/ 2019-12-22T22:17:18.767 INFO:teuthology.task.install.rpm:Package version is 12.2.12-835.ga6ab182 2019-12-22T22:17:18.826 INFO:teuthology.orchestra.run.smithi091:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2019-12-22T22:17:18.883 INFO:teuthology.orchestra.run.smithi043:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2019-12-22T22:17:18.950 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-mgr, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, rbd-fuse, rbd-mirror, rbd-nbd, librados2, python-rados, python-rgw, python-rbd, python-cephfs on remote rpm x86_64 2019-12-22T22:17:18.951 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2019-12-22T22:17:18.951 INFO:teuthology.packaging:ref: None 2019-12-22T22:17:18.951 INFO:teuthology.packaging:tag: None 2019-12-22T22:17:18.951 INFO:teuthology.packaging:branch: luminous 2019-12-22T22:17:18.951 INFO:teuthology.packaging:sha1: 99b509ffd4dd354919b08b38ba83b998fbfdc107 2019-12-22T22:17:18.952 INFO:teuthology.orchestra.run.smithi091:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/luminous/;g' /etc/yum.repos.d/ceph.repo ; fi 2019-12-22T22:17:19.009 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-mgr, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, rbd-fuse, rbd-mirror, rbd-nbd, librados2, python-rados, python-rgw, python-rbd, python-cephfs on remote rpm x86_64 2019-12-22T22:17:19.010 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2019-12-22T22:17:19.010 INFO:teuthology.packaging:ref: None 2019-12-22T22:17:19.010 INFO:teuthology.packaging:tag: None 2019-12-22T22:17:19.010 INFO:teuthology.packaging:branch: luminous 2019-12-22T22:17:19.010 INFO:teuthology.packaging:sha1: 99b509ffd4dd354919b08b38ba83b998fbfdc107 2019-12-22T22:17:19.010 INFO:teuthology.orchestra.run.smithi043:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/luminous/;g' /etc/yum.repos.d/ceph.repo ; fi 2019-12-22T22:17:19.015 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/luminous/a6ab1827d7a5dab1ee7562a6de0ae213598c4a94/centos/7/flavors/default/ 2019-12-22T22:17:19.016 INFO:teuthology.task.install.rpm:Package version is 12.2.12-835.ga6ab182 2019-12-22T22:17:19.055 INFO:teuthology.orchestra.run.smithi091:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2019-12-22T22:17:19.119 INFO:teuthology.orchestra.run.smithi043:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2019-12-22T22:17:19.138 INFO:teuthology.orchestra.run.smithi107:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2019-12-22T22:17:19.165 INFO:teuthology.orchestra.run.smithi091:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2019-12-22T22:17:19.237 INFO:teuthology.orchestra.run.smithi091.stdout:check_obsoletes = 1 2019-12-22T22:17:19.238 INFO:teuthology.orchestra.run.smithi091:> sudo yum clean all 2019-12-22T22:17:19.262 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-mgr, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, rbd-fuse, rbd-mirror, rbd-nbd, librados2, python-rados, python-rgw, python-rbd, python-cephfs on remote rpm x86_64 2019-12-22T22:17:19.262 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2019-12-22T22:17:19.263 INFO:teuthology.packaging:ref: None 2019-12-22T22:17:19.263 INFO:teuthology.packaging:tag: None 2019-12-22T22:17:19.263 INFO:teuthology.packaging:branch: luminous 2019-12-22T22:17:19.263 INFO:teuthology.packaging:sha1: 99b509ffd4dd354919b08b38ba83b998fbfdc107 2019-12-22T22:17:19.263 INFO:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/luminous/;g' /etc/yum.repos.d/ceph.repo ; fi 2019-12-22T22:17:19.271 INFO:teuthology.orchestra.run.smithi043:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2019-12-22T22:17:19.374 INFO:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2019-12-22T22:17:19.378 INFO:teuthology.orchestra.run.smithi043.stdout:check_obsoletes = 1 2019-12-22T22:17:19.380 INFO:teuthology.orchestra.run.smithi043:> sudo yum clean all 2019-12-22T22:17:19.399 INFO:teuthology.orchestra.run.smithi091.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:17:19.433 INFO:teuthology.orchestra.run.smithi091.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2019-12-22T22:17:19.433 INFO:teuthology.orchestra.run.smithi091.stdout: : lab-extras updates 2019-12-22T22:17:19.479 INFO:teuthology.orchestra.run.smithi091.stdout:Cleaning up list of fastest mirrors 2019-12-22T22:17:19.479 INFO:teuthology.orchestra.run.smithi091.stdout:Other repos take up 1.5 M of disk space (use --verbose for details) 2019-12-22T22:17:19.492 INFO:teuthology.orchestra.run.smithi091:> sudo yum -y install ceph-radosgw 2019-12-22T22:17:19.500 INFO:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2019-12-22T22:17:19.577 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2019-12-22T22:17:19.578 INFO:teuthology.orchestra.run.smithi107:> sudo yum clean all 2019-12-22T22:17:19.606 INFO:teuthology.orchestra.run.smithi043.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:17:19.663 INFO:teuthology.orchestra.run.smithi091.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:17:19.663 INFO:teuthology.orchestra.run.smithi043.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2019-12-22T22:17:19.664 INFO:teuthology.orchestra.run.smithi043.stdout: : lab-extras updates 2019-12-22T22:17:19.691 INFO:teuthology.orchestra.run.smithi091.stdout:Determining fastest mirrors 2019-12-22T22:17:19.700 INFO:teuthology.orchestra.run.smithi043.stdout:Cleaning up list of fastest mirrors 2019-12-22T22:17:19.701 INFO:teuthology.orchestra.run.smithi043.stdout:Other repos take up 1.5 M of disk space (use --verbose for details) 2019-12-22T22:17:19.710 INFO:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-radosgw 2019-12-22T22:17:19.753 INFO:teuthology.orchestra.run.smithi107.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:17:19.787 INFO:teuthology.orchestra.run.smithi107.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2019-12-22T22:17:19.788 INFO:teuthology.orchestra.run.smithi107.stdout: : lab-extras updates 2019-12-22T22:17:19.842 INFO:teuthology.orchestra.run.smithi107.stdout:Cleaning up list of fastest mirrors 2019-12-22T22:17:19.842 INFO:teuthology.orchestra.run.smithi107.stdout:Other repos take up 1.5 M of disk space (use --verbose for details) 2019-12-22T22:17:19.850 INFO:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw 2019-12-22T22:17:19.881 INFO:teuthology.orchestra.run.smithi043.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:17:19.919 INFO:teuthology.orchestra.run.smithi043.stdout:Determining fastest mirrors 2019-12-22T22:17:20.058 INFO:teuthology.orchestra.run.smithi107.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:17:20.096 INFO:teuthology.orchestra.run.smithi107.stdout:Determining fastest mirrors 2019-12-22T22:17:20.215 INFO:teuthology.orchestra.run.smithi091.stdout: * base: mirror.umd.edu 2019-12-22T22:17:20.225 INFO:teuthology.orchestra.run.smithi091.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:17:20.225 INFO:teuthology.orchestra.run.smithi091.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:17:20.225 INFO:teuthology.orchestra.run.smithi091.stdout: * updates: repos-va.psychz.net 2019-12-22T22:17:20.454 INFO:teuthology.orchestra.run.smithi107.stdout: * base: mirror.umd.edu 2019-12-22T22:17:20.455 INFO:teuthology.orchestra.run.smithi107.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:17:20.455 INFO:teuthology.orchestra.run.smithi107.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:17:20.455 INFO:teuthology.orchestra.run.smithi107.stdout: * updates: repos-va.psychz.net 2019-12-22T22:17:20.521 INFO:teuthology.orchestra.run.smithi043.stdout: * base: mirror.umd.edu 2019-12-22T22:17:20.522 INFO:teuthology.orchestra.run.smithi043.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:17:20.523 INFO:teuthology.orchestra.run.smithi043.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:17:20.523 INFO:teuthology.orchestra.run.smithi043.stdout: * updates: repos-va.psychz.net 2019-12-22T22:17:28.192 INFO:teuthology.orchestra.run.smithi091.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:17:28.469 INFO:teuthology.orchestra.run.smithi043.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:17:28.540 INFO:teuthology.orchestra.run.smithi091.stdout:Resolving Dependencies 2019-12-22T22:17:28.541 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:17:28.541 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package ceph-radosgw.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:28.554 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: librados2 = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.665 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: librgw2 = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.666 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: ceph-base = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.666 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: ceph-selinux = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.674 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: libceph-common.so.0()(64bit) for package: 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.674 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:17:28.674 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package ceph-base.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:28.676 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: librbd1 = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.676 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: libcephfs2 = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.676 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: ceph-common = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.678 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-requests for package: 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.680 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.680 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.681 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package ceph-selinux.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:28.684 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2019-12-22T22:17:28.684 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package librados2.x86_64 2:12.2.12-835.ga6ab182.el7 will be an update 2019-12-22T22:17:28.687 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package librgw2.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:28.688 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:17:28.688 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package ceph-common.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:28.692 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-rados = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.693 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-rgw = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.694 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-cephfs = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.695 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-rbd = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.696 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.697 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.697 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.698 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.698 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2019-12-22T22:17:28.699 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package libcephfs2.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:28.699 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2019-12-22T22:17:28.700 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package librbd1.x86_64 2:12.2.12-835.ga6ab182.el7 will be an update 2019-12-22T22:17:28.700 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2019-12-22T22:17:28.701 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-22T22:17:28.702 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-22T22:17:28.702 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-requests.noarch 0:2.6.0-8.el7_7 will be installed 2019-12-22T22:17:28.704 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-8.el7_7.noarch 2019-12-22T22:17:28.704 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:17:28.704 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2019-12-22T22:17:28.724 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package libradosstriper1.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:28.724 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-cephfs.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:28.724 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2019-12-22T22:17:28.724 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-rados.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:28.724 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-rbd.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:28.724 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-rgw.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:28.725 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2019-12-22T22:17:28.725 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2019-12-22T22:17:28.836 INFO:teuthology.orchestra.run.smithi043.stdout:Resolving Dependencies 2019-12-22T22:17:28.837 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:17:28.837 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package ceph-radosgw.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:28.852 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: librados2 = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.963 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: librgw2 = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.964 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: ceph-base = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.966 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: ceph-selinux = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.967 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: libceph-common.so.0()(64bit) for package: 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.968 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:17:28.968 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package ceph-base.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:28.978 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: librbd1 = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.979 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: libcephfs2 = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.981 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: ceph-common = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.982 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-requests for package: 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.985 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.986 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.987 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package ceph-selinux.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:28.989 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2019-12-22T22:17:28.990 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package librados2.x86_64 2:12.2.12-835.ga6ab182.el7 will be an update 2019-12-22T22:17:28.993 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package librgw2.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:28.994 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:17:28.995 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package ceph-common.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:28.998 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-rados = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:28.999 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-rgw = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:29.000 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-cephfs = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:29.001 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-rbd = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:29.002 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:29.003 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:29.005 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:29.006 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:29.006 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2019-12-22T22:17:29.006 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package libcephfs2.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:29.007 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2019-12-22T22:17:29.007 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package librbd1.x86_64 2:12.2.12-835.ga6ab182.el7 will be an update 2019-12-22T22:17:29.008 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2019-12-22T22:17:29.008 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-22T22:17:29.012 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-22T22:17:29.013 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-requests.noarch 0:2.6.0-8.el7_7 will be installed 2019-12-22T22:17:29.013 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-8.el7_7.noarch 2019-12-22T22:17:29.013 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:17:29.013 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2019-12-22T22:17:29.014 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package libradosstriper1.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:29.014 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-cephfs.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:29.016 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2019-12-22T22:17:29.016 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-rados.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:29.016 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-rbd.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:29.017 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-rgw.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:29.017 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2019-12-22T22:17:29.017 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2019-12-22T22:17:29.564 INFO:teuthology.orchestra.run.smithi091.stdout:--> Finished Dependency Resolution 2019-12-22T22:17:29.648 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:17:29.648 INFO:teuthology.orchestra.run.smithi091.stdout:Dependencies Resolved 2019-12-22T22:17:29.651 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:17:29.651 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:17:29.652 INFO:teuthology.orchestra.run.smithi091.stdout: Package Arch Version Repository Size 2019-12-22T22:17:29.652 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:17:29.652 INFO:teuthology.orchestra.run.smithi091.stdout:Installing: 2019-12-22T22:17:29.652 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-radosgw x86_64 2:12.2.12-835.ga6ab182.el7 ceph 4.0 M 2019-12-22T22:17:29.653 INFO:teuthology.orchestra.run.smithi091.stdout:Installing for dependencies: 2019-12-22T22:17:29.653 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-base x86_64 2:12.2.12-835.ga6ab182.el7 ceph 4.1 M 2019-12-22T22:17:29.653 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-common x86_64 2:12.2.12-835.ga6ab182.el7 ceph 16 M 2019-12-22T22:17:29.654 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-selinux x86_64 2:12.2.12-835.ga6ab182.el7 ceph 19 k 2019-12-22T22:17:29.654 INFO:teuthology.orchestra.run.smithi091.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2019-12-22T22:17:29.655 INFO:teuthology.orchestra.run.smithi091.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2019-12-22T22:17:29.655 INFO:teuthology.orchestra.run.smithi091.stdout: libcephfs2 x86_64 2:12.2.12-835.ga6ab182.el7 ceph 438 k 2019-12-22T22:17:29.655 INFO:teuthology.orchestra.run.smithi091.stdout: libradosstriper1 x86_64 2:12.2.12-835.ga6ab182.el7 ceph 331 k 2019-12-22T22:17:29.656 INFO:teuthology.orchestra.run.smithi091.stdout: librgw2 x86_64 2:12.2.12-835.ga6ab182.el7 ceph 1.8 M 2019-12-22T22:17:29.657 INFO:teuthology.orchestra.run.smithi091.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2019-12-22T22:17:29.657 INFO:teuthology.orchestra.run.smithi091.stdout: python-cephfs x86_64 2:12.2.12-835.ga6ab182.el7 ceph 83 k 2019-12-22T22:17:29.657 INFO:teuthology.orchestra.run.smithi091.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2019-12-22T22:17:29.657 INFO:teuthology.orchestra.run.smithi091.stdout: python-rados x86_64 2:12.2.12-835.ga6ab182.el7 ceph 174 k 2019-12-22T22:17:29.658 INFO:teuthology.orchestra.run.smithi091.stdout: python-rbd x86_64 2:12.2.12-835.ga6ab182.el7 ceph 105 k 2019-12-22T22:17:29.658 INFO:teuthology.orchestra.run.smithi091.stdout: python-requests noarch 2.6.0-8.el7_7 updates 95 k 2019-12-22T22:17:29.658 INFO:teuthology.orchestra.run.smithi091.stdout: python-rgw x86_64 2:12.2.12-835.ga6ab182.el7 ceph 72 k 2019-12-22T22:17:29.658 INFO:teuthology.orchestra.run.smithi091.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2019-12-22T22:17:29.659 INFO:teuthology.orchestra.run.smithi091.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2019-12-22T22:17:29.659 INFO:teuthology.orchestra.run.smithi091.stdout:Updating for dependencies: 2019-12-22T22:17:29.659 INFO:teuthology.orchestra.run.smithi091.stdout: librados2 x86_64 2:12.2.12-835.ga6ab182.el7 ceph 2.9 M 2019-12-22T22:17:29.659 INFO:teuthology.orchestra.run.smithi091.stdout: librbd1 x86_64 2:12.2.12-835.ga6ab182.el7 ceph 1.1 M 2019-12-22T22:17:29.660 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:17:29.660 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction Summary 2019-12-22T22:17:29.660 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:17:29.660 INFO:teuthology.orchestra.run.smithi091.stdout:Install 1 Package (+17 Dependent packages) 2019-12-22T22:17:29.660 INFO:teuthology.orchestra.run.smithi091.stdout:Upgrade ( 2 Dependent packages) 2019-12-22T22:17:29.661 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:17:29.661 INFO:teuthology.orchestra.run.smithi091.stdout:Total download size: 31 M 2019-12-22T22:17:29.661 INFO:teuthology.orchestra.run.smithi091.stdout:Downloading packages: 2019-12-22T22:17:29.662 INFO:teuthology.orchestra.run.smithi091.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-22T22:17:29.922 INFO:teuthology.orchestra.run.smithi043.stdout:--> Finished Dependency Resolution 2019-12-22T22:17:30.010 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:17:30.010 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies Resolved 2019-12-22T22:17:30.013 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:17:30.014 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:17:30.014 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2019-12-22T22:17:30.014 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:17:30.014 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2019-12-22T22:17:30.014 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw x86_64 2:12.2.12-835.ga6ab182.el7 ceph 4.0 M 2019-12-22T22:17:30.014 INFO:teuthology.orchestra.run.smithi043.stdout:Installing for dependencies: 2019-12-22T22:17:30.014 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base x86_64 2:12.2.12-835.ga6ab182.el7 ceph 4.1 M 2019-12-22T22:17:30.015 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common x86_64 2:12.2.12-835.ga6ab182.el7 ceph 16 M 2019-12-22T22:17:30.015 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux x86_64 2:12.2.12-835.ga6ab182.el7 ceph 19 k 2019-12-22T22:17:30.015 INFO:teuthology.orchestra.run.smithi043.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2019-12-22T22:17:30.015 INFO:teuthology.orchestra.run.smithi043.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2019-12-22T22:17:30.015 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 x86_64 2:12.2.12-835.ga6ab182.el7 ceph 438 k 2019-12-22T22:17:30.015 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 x86_64 2:12.2.12-835.ga6ab182.el7 ceph 331 k 2019-12-22T22:17:30.015 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2 x86_64 2:12.2.12-835.ga6ab182.el7 ceph 1.8 M 2019-12-22T22:17:30.016 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2019-12-22T22:17:30.016 INFO:teuthology.orchestra.run.smithi043.stdout: python-cephfs x86_64 2:12.2.12-835.ga6ab182.el7 ceph 83 k 2019-12-22T22:17:30.016 INFO:teuthology.orchestra.run.smithi043.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2019-12-22T22:17:30.016 INFO:teuthology.orchestra.run.smithi043.stdout: python-rados x86_64 2:12.2.12-835.ga6ab182.el7 ceph 174 k 2019-12-22T22:17:30.016 INFO:teuthology.orchestra.run.smithi043.stdout: python-rbd x86_64 2:12.2.12-835.ga6ab182.el7 ceph 105 k 2019-12-22T22:17:30.016 INFO:teuthology.orchestra.run.smithi043.stdout: python-requests noarch 2.6.0-8.el7_7 updates 95 k 2019-12-22T22:17:30.016 INFO:teuthology.orchestra.run.smithi043.stdout: python-rgw x86_64 2:12.2.12-835.ga6ab182.el7 ceph 72 k 2019-12-22T22:17:30.017 INFO:teuthology.orchestra.run.smithi043.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2019-12-22T22:17:30.017 INFO:teuthology.orchestra.run.smithi043.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2019-12-22T22:17:30.017 INFO:teuthology.orchestra.run.smithi043.stdout:Updating for dependencies: 2019-12-22T22:17:30.017 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 x86_64 2:12.2.12-835.ga6ab182.el7 ceph 2.9 M 2019-12-22T22:17:30.017 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1 x86_64 2:12.2.12-835.ga6ab182.el7 ceph 1.1 M 2019-12-22T22:17:30.017 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:17:30.018 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2019-12-22T22:17:30.018 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:17:30.018 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package (+17 Dependent packages) 2019-12-22T22:17:30.018 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade ( 2 Dependent packages) 2019-12-22T22:17:30.018 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:17:30.019 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 31 M 2019-12-22T22:17:30.019 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading packages: 2019-12-22T22:17:30.019 INFO:teuthology.orchestra.run.smithi043.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-22T22:17:30.256 INFO:teuthology.orchestra.run.smithi107.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:17:30.609 INFO:teuthology.orchestra.run.smithi107.stdout:Resolving Dependencies 2019-12-22T22:17:30.609 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:17:30.612 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package ceph-radosgw.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:30.625 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: librados2 = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:30.730 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: librgw2 = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:30.732 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: ceph-base = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:30.733 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: ceph-selinux = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:30.736 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: libceph-common.so.0()(64bit) for package: 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:30.737 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:17:30.737 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package ceph-base.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:30.744 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: librbd1 = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:30.745 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: libcephfs2 = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:30.753 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: ceph-common = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:30.753 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-requests for package: 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:30.754 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:30.754 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:30.754 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package ceph-selinux.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:30.754 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2019-12-22T22:17:30.755 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package librados2.x86_64 2:12.2.12-835.ga6ab182.el7 will be an update 2019-12-22T22:17:30.757 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package librgw2.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:30.758 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:17:30.758 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package ceph-common.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:30.762 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-rados = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:30.764 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-rgw = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:30.764 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-cephfs = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:30.765 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-rbd = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:30.768 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:30.768 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:30.768 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:30.769 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:30.769 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2019-12-22T22:17:30.769 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package libcephfs2.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:30.770 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2019-12-22T22:17:30.770 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package librbd1.x86_64 2:12.2.12-835.ga6ab182.el7 will be an update 2019-12-22T22:17:30.771 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2019-12-22T22:17:30.772 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-22T22:17:30.773 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-22T22:17:30.773 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-requests.noarch 0:2.6.0-8.el7_7 will be installed 2019-12-22T22:17:30.774 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-8.el7_7.noarch 2019-12-22T22:17:30.774 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:17:30.775 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2019-12-22T22:17:30.776 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package libradosstriper1.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:30.777 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-cephfs.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:30.778 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2019-12-22T22:17:30.778 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-rados.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:30.779 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-rbd.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:30.779 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-rgw.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:30.779 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2019-12-22T22:17:30.780 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2019-12-22T22:17:31.633 INFO:teuthology.orchestra.run.smithi107.stdout:--> Finished Dependency Resolution 2019-12-22T22:17:31.718 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:17:31.718 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies Resolved 2019-12-22T22:17:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:17:31.721 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:17:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2019-12-22T22:17:31.721 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:17:31.721 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2019-12-22T22:17:31.721 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:12.2.12-835.ga6ab182.el7 ceph 4.0 M 2019-12-22T22:17:31.722 INFO:teuthology.orchestra.run.smithi107.stdout:Installing for dependencies: 2019-12-22T22:17:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:12.2.12-835.ga6ab182.el7 ceph 4.1 M 2019-12-22T22:17:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:12.2.12-835.ga6ab182.el7 ceph 16 M 2019-12-22T22:17:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:12.2.12-835.ga6ab182.el7 ceph 19 k 2019-12-22T22:17:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2019-12-22T22:17:31.722 INFO:teuthology.orchestra.run.smithi107.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2019-12-22T22:17:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:12.2.12-835.ga6ab182.el7 ceph 438 k 2019-12-22T22:17:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:12.2.12-835.ga6ab182.el7 ceph 331 k 2019-12-22T22:17:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:12.2.12-835.ga6ab182.el7 ceph 1.8 M 2019-12-22T22:17:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2019-12-22T22:17:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: python-cephfs x86_64 2:12.2.12-835.ga6ab182.el7 ceph 83 k 2019-12-22T22:17:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2019-12-22T22:17:31.723 INFO:teuthology.orchestra.run.smithi107.stdout: python-rados x86_64 2:12.2.12-835.ga6ab182.el7 ceph 174 k 2019-12-22T22:17:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: python-rbd x86_64 2:12.2.12-835.ga6ab182.el7 ceph 105 k 2019-12-22T22:17:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: python-requests noarch 2.6.0-8.el7_7 updates 95 k 2019-12-22T22:17:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: python-rgw x86_64 2:12.2.12-835.ga6ab182.el7 ceph 72 k 2019-12-22T22:17:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2019-12-22T22:17:31.724 INFO:teuthology.orchestra.run.smithi107.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2019-12-22T22:17:31.724 INFO:teuthology.orchestra.run.smithi107.stdout:Updating for dependencies: 2019-12-22T22:17:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:12.2.12-835.ga6ab182.el7 ceph 2.9 M 2019-12-22T22:17:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:12.2.12-835.ga6ab182.el7 ceph 1.1 M 2019-12-22T22:17:31.725 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:17:31.725 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2019-12-22T22:17:31.725 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:17:31.725 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package (+17 Dependent packages) 2019-12-22T22:17:31.726 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade ( 2 Dependent packages) 2019-12-22T22:17:31.726 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:17:31.726 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 31 M 2019-12-22T22:17:31.726 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading packages: 2019-12-22T22:17:31.726 INFO:teuthology.orchestra.run.smithi107.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-22T22:17:34.935 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2019-12-22T22:17:34.936 INFO:teuthology.orchestra.run.smithi107.stdout:Total 9.8 MB/s | 31 MB 00:03 2019-12-22T22:17:34.952 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2019-12-22T22:17:35.022 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2019-12-22T22:17:35.082 INFO:teuthology.orchestra.run.smithi091.stdout:-------------------------------------------------------------------------------- 2019-12-22T22:17:35.082 INFO:teuthology.orchestra.run.smithi091.stdout:Total 5.8 MB/s | 31 MB 00:05 2019-12-22T22:17:35.094 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction check 2019-12-22T22:17:35.150 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction test 2019-12-22T22:17:35.258 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded 2019-12-22T22:17:35.259 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2019-12-22T22:17:35.340 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction test succeeded 2019-12-22T22:17:35.346 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction 2019-12-22T22:17:36.094 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 1/22 2019-12-22T22:17:36.138 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 1/22 2019-12-22T22:17:36.350 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2019-12-22T22:17:36.350 INFO:teuthology.orchestra.run.smithi043.stdout:Total 4.9 MB/s | 31 MB 00:06 2019-12-22T22:17:36.361 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2019-12-22T22:17:36.411 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2019-12-22T22:17:36.502 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 2/22 2019-12-22T22:17:36.525 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 2/22 2019-12-22T22:17:36.630 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded 2019-12-22T22:17:36.630 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2019-12-22T22:17:37.213 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 3/22 2019-12-22T22:17:37.250 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 3/22 2019-12-22T22:17:37.395 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 1/22 2019-12-22T22:17:37.590 INFO:teuthology.orchestra.run.smithi107.stdout: Updating : 2:librados2-12.2.12-835.ga6ab182.el7.x86_64 4/22 2019-12-22T22:17:37.614 INFO:teuthology.orchestra.run.smithi091.stdout: Updating : 2:librados2-12.2.12-835.ga6ab182.el7.x86_64 4/22 2019-12-22T22:17:37.798 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 2/22 2019-12-22T22:17:38.097 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:python-rados-12.2.12-835.ga6ab182.el7.x86_64 5/22 2019-12-22T22:17:38.104 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:python-rados-12.2.12-835.ga6ab182.el7.x86_64 5/22 2019-12-22T22:17:38.481 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:librgw2-12.2.12-835.ga6ab182.el7.x86_64 6/22 2019-12-22T22:17:38.487 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:librgw2-12.2.12-835.ga6ab182.el7.x86_64 6/22 2019-12-22T22:17:38.518 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 3/22 2019-12-22T22:17:38.890 INFO:teuthology.orchestra.run.smithi043.stdout: Updating : 2:librados2-12.2.12-835.ga6ab182.el7.x86_64 4/22 2019-12-22T22:17:38.967 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:libcephfs2-12.2.12-835.ga6ab182.el7.x86_64 7/22 2019-12-22T22:17:39.012 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:libcephfs2-12.2.12-835.ga6ab182.el7.x86_64 7/22 2019-12-22T22:17:39.277 INFO:teuthology.orchestra.run.smithi107.stdout: Updating : 2:librbd1-12.2.12-835.ga6ab182.el7.x86_64 8/22 2019-12-22T22:17:39.354 INFO:teuthology.orchestra.run.smithi091.stdout: Updating : 2:librbd1-12.2.12-835.ga6ab182.el7.x86_64 8/22 2019-12-22T22:17:39.561 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:python-rbd-12.2.12-835.ga6ab182.el7.x86_64 9/22 2019-12-22T22:17:39.660 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:python-rbd-12.2.12-835.ga6ab182.el7.x86_64 9/22 2019-12-22T22:17:39.835 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:python-rados-12.2.12-835.ga6ab182.el7.x86_64 5/22 2019-12-22T22:17:39.887 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:python-cephfs-12.2.12-835.ga6ab182.el7.x86_64 10/22 2019-12-22T22:17:39.971 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:python-cephfs-12.2.12-835.ga6ab182.el7.x86_64 10/22 2019-12-22T22:17:40.159 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:librgw2-12.2.12-835.ga6ab182.el7.x86_64 6/22 2019-12-22T22:17:40.225 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:python-rgw-12.2.12-835.ga6ab182.el7.x86_64 11/22 2019-12-22T22:17:40.305 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:python-rgw-12.2.12-835.ga6ab182.el7.x86_64 11/22 2019-12-22T22:17:40.521 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:libradosstriper1-12.2.12-835.ga6ab182.el7.x86_64 12/22 2019-12-22T22:17:40.563 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:libradosstriper1-12.2.12-835.ga6ab182.el7.x86_64 12/22 2019-12-22T22:17:40.626 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:libcephfs2-12.2.12-835.ga6ab182.el7.x86_64 7/22 2019-12-22T22:17:40.794 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 13/22 2019-12-22T22:17:40.835 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 13/22 2019-12-22T22:17:41.056 INFO:teuthology.orchestra.run.smithi043.stdout: Updating : 2:librbd1-12.2.12-835.ga6ab182.el7.x86_64 8/22 2019-12-22T22:17:41.160 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 14/22 2019-12-22T22:17:41.234 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 14/22 2019-12-22T22:17:41.347 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:python-rbd-12.2.12-835.ga6ab182.el7.x86_64 9/22 2019-12-22T22:17:41.510 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 15/22 2019-12-22T22:17:41.583 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 15/22 2019-12-22T22:17:41.648 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:python-cephfs-12.2.12-835.ga6ab182.el7.x86_64 10/22 2019-12-22T22:17:41.993 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:python-rgw-12.2.12-835.ga6ab182.el7.x86_64 11/22 2019-12-22T22:17:42.262 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:libradosstriper1-12.2.12-835.ga6ab182.el7.x86_64 12/22 2019-12-22T22:17:42.527 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 13/22 2019-12-22T22:17:42.929 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 14/22 2019-12-22T22:17:43.213 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 15/22 2019-12-22T22:17:43.714 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python-requests-2.6.0-8.el7_7.noarch 16/22 2019-12-22T22:17:43.776 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python-requests-2.6.0-8.el7_7.noarch 16/22 2019-12-22T22:17:44.716 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 17/22 2019-12-22T22:17:44.805 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 17/22 2019-12-22T22:17:45.313 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 18/22 2019-12-22T22:17:45.380 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python-requests-2.6.0-8.el7_7.noarch 16/22 2019-12-22T22:17:45.380 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 18/22 2019-12-22T22:17:46.351 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 17/22 2019-12-22T22:17:46.937 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 18/22 2019-12-22T22:17:54.239 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:ceph-selinux-12.2.12-835.ga6ab182.el7.x86_64 19/22 2019-12-22T22:17:54.293 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:ceph-selinux-12.2.12-835.ga6ab182.el7.x86_64 19/22 2019-12-22T22:17:54.618 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 20/22 2019-12-22T22:17:54.743 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 20/22 2019-12-22T22:17:54.893 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 21/22 2019-12-22T22:17:55.029 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 21/22 2019-12-22T22:17:55.191 INFO:teuthology.orchestra.run.smithi091.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 22/22 2019-12-22T22:17:55.320 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:librgw2-12.2.12-835.ga6ab182.el7.x86_64 1/22 2019-12-22T22:17:55.345 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 22/22 2019-12-22T22:17:55.465 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 2/22 2019-12-22T22:17:55.486 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:librgw2-12.2.12-835.ga6ab182.el7.x86_64 1/22 2019-12-22T22:17:55.582 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 3/22 2019-12-22T22:17:55.616 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 2/22 2019-12-22T22:17:55.725 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:ceph-selinux-12.2.12-835.ga6ab182.el7.x86_64 4/22 2019-12-22T22:17:55.760 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 3/22 2019-12-22T22:17:55.837 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:libcephfs2-12.2.12-835.ga6ab182.el7.x86_64 5/22 2019-12-22T22:17:55.858 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:ceph-selinux-12.2.12-835.ga6ab182.el7.x86_64 4/22 2019-12-22T22:17:55.943 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:libcephfs2-12.2.12-835.ga6ab182.el7.x86_64 5/22 2019-12-22T22:17:55.951 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 6/22 2019-12-22T22:17:56.068 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 6/22 2019-12-22T22:17:56.101 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 7/22 2019-12-22T22:17:56.161 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 7/22 2019-12-22T22:17:56.202 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:librados2-12.2.12-835.ga6ab182.el7.x86_64 8/22 2019-12-22T22:17:56.282 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:librados2-12.2.12-835.ga6ab182.el7.x86_64 8/22 2019-12-22T22:17:56.336 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:python-rbd-12.2.12-835.ga6ab182.el7.x86_64 9/22 2019-12-22T22:17:56.386 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:python-rbd-12.2.12-835.ga6ab182.el7.x86_64 9/22 2019-12-22T22:17:56.389 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:ceph-selinux-12.2.12-835.ga6ab182.el7.x86_64 19/22 2019-12-22T22:17:56.420 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 10/22 2019-12-22T22:17:56.512 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:python-cephfs-12.2.12-835.ga6ab182.el7.x86_64 11/22 2019-12-22T22:17:56.553 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 10/22 2019-12-22T22:17:56.597 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:libradosstriper1-12.2.12-835.ga6ab182.el7.x86_64 12/22 2019-12-22T22:17:56.680 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:python-cephfs-12.2.12-835.ga6ab182.el7.x86_64 11/22 2019-12-22T22:17:56.721 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:python-rgw-12.2.12-835.ga6ab182.el7.x86_64 13/22 2019-12-22T22:17:56.785 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 20/22 2019-12-22T22:17:56.816 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:libradosstriper1-12.2.12-835.ga6ab182.el7.x86_64 12/22 2019-12-22T22:17:56.838 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 14/22 2019-12-22T22:17:56.930 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:python-rgw-12.2.12-835.ga6ab182.el7.x86_64 13/22 2019-12-22T22:17:56.989 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python-requests-2.6.0-8.el7_7.noarch 15/22 2019-12-22T22:17:57.031 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 21/22 2019-12-22T22:17:57.078 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 14/22 2019-12-22T22:17:57.090 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 16/22 2019-12-22T22:17:57.191 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 17/22 2019-12-22T22:17:57.215 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python-requests-2.6.0-8.el7_7.noarch 15/22 2019-12-22T22:17:57.319 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 22/22 2019-12-22T22:17:57.332 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:librbd1-12.2.12-835.ga6ab182.el7.x86_64 18/22 2019-12-22T22:17:57.341 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 16/22 2019-12-22T22:17:57.433 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:librgw2-12.2.12-835.ga6ab182.el7.x86_64 1/22 2019-12-22T22:17:57.448 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 17/22 2019-12-22T22:17:57.449 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:python-rados-12.2.12-835.ga6ab182.el7.x86_64 19/22 2019-12-22T22:17:57.566 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 2/22 2019-12-22T22:17:57.597 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 20/22 2019-12-22T22:17:57.602 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 21/22 2019-12-22T22:17:57.675 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:ceph-common-12.2.12-835.ga6ab182.el7.x86_64 3/22 2019-12-22T22:17:57.818 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:ceph-selinux-12.2.12-835.ga6ab182.el7.x86_64 4/22 2019-12-22T22:17:57.918 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:libcephfs2-12.2.12-835.ga6ab182.el7.x86_64 5/22 2019-12-22T22:17:57.984 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 22/22 2019-12-22T22:17:57.984 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:17:57.984 INFO:teuthology.orchestra.run.smithi091.stdout:Installed: 2019-12-22T22:17:57.985 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-radosgw.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:57.985 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:17:57.985 INFO:teuthology.orchestra.run.smithi091.stdout:Dependency Installed: 2019-12-22T22:17:57.985 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-base.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:57.985 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-common.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:57.985 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-selinux.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:57.985 INFO:teuthology.orchestra.run.smithi091.stdout: leveldb.x86_64 0:1.12.0-11.el7 2019-12-22T22:17:57.986 INFO:teuthology.orchestra.run.smithi091.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2019-12-22T22:17:57.986 INFO:teuthology.orchestra.run.smithi091.stdout: libcephfs2.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:57.986 INFO:teuthology.orchestra.run.smithi091.stdout: libradosstriper1.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:57.986 INFO:teuthology.orchestra.run.smithi091.stdout: librgw2.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:57.986 INFO:teuthology.orchestra.run.smithi091.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2019-12-22T22:17:57.986 INFO:teuthology.orchestra.run.smithi091.stdout: python-cephfs.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:57.986 INFO:teuthology.orchestra.run.smithi091.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2019-12-22T22:17:57.987 INFO:teuthology.orchestra.run.smithi091.stdout: python-rados.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:57.987 INFO:teuthology.orchestra.run.smithi091.stdout: python-rbd.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:57.987 INFO:teuthology.orchestra.run.smithi091.stdout: python-requests.noarch 0:2.6.0-8.el7_7 2019-12-22T22:17:57.987 INFO:teuthology.orchestra.run.smithi091.stdout: python-rgw.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:57.987 INFO:teuthology.orchestra.run.smithi091.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2019-12-22T22:17:57.987 INFO:teuthology.orchestra.run.smithi091.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2019-12-22T22:17:57.987 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:17:57.988 INFO:teuthology.orchestra.run.smithi091.stdout:Dependency Updated: 2019-12-22T22:17:57.988 INFO:teuthology.orchestra.run.smithi091.stdout: librados2.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:57.988 INFO:teuthology.orchestra.run.smithi091.stdout: librbd1.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:57.988 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:17:57.988 INFO:teuthology.orchestra.run.smithi091.stdout:Complete! 2019-12-22T22:17:58.018 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 6/22 2019-12-22T22:17:58.120 INFO:teuthology.orchestra.run.smithi091:> sudo yum -y install ceph-test 2019-12-22T22:17:58.136 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 7/22 2019-12-22T22:17:58.278 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:librados2-12.2.12-835.ga6ab182.el7.x86_64 8/22 2019-12-22T22:17:58.387 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:python-rbd-12.2.12-835.ga6ab182.el7.x86_64 9/22 2019-12-22T22:17:58.496 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 10/22 2019-12-22T22:17:58.541 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:librbd1-12.2.12-835.ga6ab182.el7.x86_64 18/22 2019-12-22T22:17:58.569 INFO:teuthology.orchestra.run.smithi091.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:17:58.588 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:python-cephfs-12.2.12-835.ga6ab182.el7.x86_64 11/22 2019-12-22T22:17:58.608 INFO:teuthology.orchestra.run.smithi091.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:17:58.609 INFO:teuthology.orchestra.run.smithi091.stdout: * base: mirror.umd.edu 2019-12-22T22:17:58.610 INFO:teuthology.orchestra.run.smithi091.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:17:58.610 INFO:teuthology.orchestra.run.smithi091.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:17:58.610 INFO:teuthology.orchestra.run.smithi091.stdout: * updates: repos-va.psychz.net 2019-12-22T22:17:58.672 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:libradosstriper1-12.2.12-835.ga6ab182.el7.x86_64 12/22 2019-12-22T22:17:58.674 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:python-rados-12.2.12-835.ga6ab182.el7.x86_64 19/22 2019-12-22T22:17:58.767 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 20/22 2019-12-22T22:17:58.772 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 21/22 2019-12-22T22:17:58.797 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:python-rgw-12.2.12-835.ga6ab182.el7.x86_64 13/22 2019-12-22T22:17:58.898 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 14/22 2019-12-22T22:17:59.015 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python-requests-2.6.0-8.el7_7.noarch 15/22 2019-12-22T22:17:59.101 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 16/22 2019-12-22T22:17:59.108 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 22/22 2019-12-22T22:17:59.108 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:17:59.108 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2019-12-22T22:17:59.108 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.108 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:17:59.121 INFO:teuthology.orchestra.run.smithi107.stdout:Dependency Installed: 2019-12-22T22:17:59.122 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.122 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.122 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.122 INFO:teuthology.orchestra.run.smithi107.stdout: leveldb.x86_64 0:1.12.0-11.el7 2019-12-22T22:17:59.122 INFO:teuthology.orchestra.run.smithi107.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2019-12-22T22:17:59.122 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.123 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.133 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.133 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2019-12-22T22:17:59.133 INFO:teuthology.orchestra.run.smithi107.stdout: python-cephfs.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.133 INFO:teuthology.orchestra.run.smithi107.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2019-12-22T22:17:59.133 INFO:teuthology.orchestra.run.smithi107.stdout: python-rados.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.134 INFO:teuthology.orchestra.run.smithi107.stdout: python-rbd.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.134 INFO:teuthology.orchestra.run.smithi107.stdout: python-requests.noarch 0:2.6.0-8.el7_7 2019-12-22T22:17:59.134 INFO:teuthology.orchestra.run.smithi107.stdout: python-rgw.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.134 INFO:teuthology.orchestra.run.smithi107.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2019-12-22T22:17:59.134 INFO:teuthology.orchestra.run.smithi107.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2019-12-22T22:17:59.134 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:17:59.134 INFO:teuthology.orchestra.run.smithi107.stdout:Dependency Updated: 2019-12-22T22:17:59.135 INFO:teuthology.orchestra.run.smithi107.stdout: librados2.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.135 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.135 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:17:59.135 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2019-12-22T22:17:59.240 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:ceph-radosgw-12.2.12-835.ga6ab182.el7.x86_64 17/22 2019-12-22T22:17:59.256 INFO:teuthology.orchestra.run.smithi091.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:17:59.268 INFO:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-test 2019-12-22T22:17:59.352 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:librbd1-12.2.12-835.ga6ab182.el7.x86_64 18/22 2019-12-22T22:17:59.469 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:python-rados-12.2.12-835.ga6ab182.el7.x86_64 19/22 2019-12-22T22:17:59.570 INFO:teuthology.orchestra.run.smithi107.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:17:59.587 INFO:teuthology.orchestra.run.smithi091.stdout:Resolving Dependencies 2019-12-22T22:17:59.587 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:17:59.587 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package ceph-test.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:17:59.595 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:ceph-base-12.2.12-835.ga6ab182.el7.x86_64 20/22 2019-12-22T22:17:59.596 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 21/22 2019-12-22T22:17:59.606 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:59.614 INFO:teuthology.orchestra.run.smithi107.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:17:59.618 INFO:teuthology.orchestra.run.smithi107.stdout: * base: mirror.umd.edu 2019-12-22T22:17:59.618 INFO:teuthology.orchestra.run.smithi107.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:17:59.618 INFO:teuthology.orchestra.run.smithi107.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:17:59.618 INFO:teuthology.orchestra.run.smithi107.stdout: * updates: repos-va.psychz.net 2019-12-22T22:17:59.635 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: jq for package: 2:ceph-test-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:59.636 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: socat for package: 2:ceph-test-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:17:59.637 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:17:59.637 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package jq.x86_64 0:1.5-1.el7 will be installed 2019-12-22T22:17:59.638 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.5-1.el7.x86_64 2019-12-22T22:17:59.639 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2019-12-22T22:17:59.640 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2019-12-22T22:17:59.643 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:17:59.643 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2019-12-22T22:17:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 22/22 2019-12-22T22:17:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:17:59.953 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2019-12-22T22:17:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.953 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:17:59.954 INFO:teuthology.orchestra.run.smithi043.stdout:Dependency Installed: 2019-12-22T22:17:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: leveldb.x86_64 0:1.12.0-11.el7 2019-12-22T22:17:59.954 INFO:teuthology.orchestra.run.smithi043.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2019-12-22T22:17:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.955 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2019-12-22T22:17:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: python-cephfs.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2019-12-22T22:17:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: python-rados.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: python-rbd.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: python-requests.noarch 0:2.6.0-8.el7_7 2019-12-22T22:17:59.956 INFO:teuthology.orchestra.run.smithi043.stdout: python-rgw.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2019-12-22T22:17:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2019-12-22T22:17:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:17:59.957 INFO:teuthology.orchestra.run.smithi043.stdout:Dependency Updated: 2019-12-22T22:17:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: librados2.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.957 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:17:59.958 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:17:59.958 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2019-12-22T22:18:00.004 INFO:teuthology.orchestra.run.smithi091.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:00.085 INFO:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-test 2019-12-22T22:18:00.226 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:00.227 INFO:teuthology.orchestra.run.smithi091.stdout:Dependencies Resolved 2019-12-22T22:18:00.227 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:00.227 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:00.228 INFO:teuthology.orchestra.run.smithi091.stdout: Package Arch Version Repository 2019-12-22T22:18:00.228 INFO:teuthology.orchestra.run.smithi091.stdout: Size 2019-12-22T22:18:00.234 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:00.234 INFO:teuthology.orchestra.run.smithi091.stdout:Installing: 2019-12-22T22:18:00.235 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-test x86_64 2:12.2.12-835.ga6ab182.el7 ceph 35 M 2019-12-22T22:18:00.235 INFO:teuthology.orchestra.run.smithi091.stdout:Installing for dependencies: 2019-12-22T22:18:00.235 INFO:teuthology.orchestra.run.smithi091.stdout: jq x86_64 1.5-1.el7 epel 153 k 2019-12-22T22:18:00.235 INFO:teuthology.orchestra.run.smithi091.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2019-12-22T22:18:00.236 INFO:teuthology.orchestra.run.smithi091.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2019-12-22T22:18:00.236 INFO:teuthology.orchestra.run.smithi091.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2019-12-22T22:18:00.237 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:00.237 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction Summary 2019-12-22T22:18:00.237 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:00.237 INFO:teuthology.orchestra.run.smithi091.stdout:Install 1 Package (+4 Dependent packages) 2019-12-22T22:18:00.238 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:00.238 INFO:teuthology.orchestra.run.smithi091.stdout:Total download size: 36 M 2019-12-22T22:18:00.238 INFO:teuthology.orchestra.run.smithi091.stdout:Installed size: 127 M 2019-12-22T22:18:00.238 INFO:teuthology.orchestra.run.smithi091.stdout:Downloading packages: 2019-12-22T22:18:00.263 INFO:teuthology.orchestra.run.smithi107.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:00.389 INFO:teuthology.orchestra.run.smithi043.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:00.437 INFO:teuthology.orchestra.run.smithi043.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: * base: mirror.umd.edu 2019-12-22T22:18:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:00.439 INFO:teuthology.orchestra.run.smithi043.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:00.440 INFO:teuthology.orchestra.run.smithi043.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:00.598 INFO:teuthology.orchestra.run.smithi107.stdout:Resolving Dependencies 2019-12-22T22:18:00.598 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:18:00.599 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package ceph-test.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:00.617 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:00.646 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: jq for package: 2:ceph-test-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:00.647 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: socat for package: 2:ceph-test-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:00.648 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:18:00.648 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package jq.x86_64 0:1.5-1.el7 will be installed 2019-12-22T22:18:00.649 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.5-1.el7.x86_64 2019-12-22T22:18:00.649 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2019-12-22T22:18:00.651 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2019-12-22T22:18:00.653 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:18:00.654 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2019-12-22T22:18:01.014 INFO:teuthology.orchestra.run.smithi107.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:01.103 INFO:teuthology.orchestra.run.smithi043.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:01.233 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:01.233 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies Resolved 2019-12-22T22:18:01.234 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:01.234 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:01.234 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository 2019-12-22T22:18:01.235 INFO:teuthology.orchestra.run.smithi107.stdout: Size 2019-12-22T22:18:01.235 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:01.235 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2019-12-22T22:18:01.235 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:12.2.12-835.ga6ab182.el7 ceph 35 M 2019-12-22T22:18:01.235 INFO:teuthology.orchestra.run.smithi107.stdout:Installing for dependencies: 2019-12-22T22:18:01.235 INFO:teuthology.orchestra.run.smithi107.stdout: jq x86_64 1.5-1.el7 epel 153 k 2019-12-22T22:18:01.235 INFO:teuthology.orchestra.run.smithi107.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2019-12-22T22:18:01.236 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2019-12-22T22:18:01.236 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2019-12-22T22:18:01.236 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:01.236 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2019-12-22T22:18:01.236 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:01.236 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package (+4 Dependent packages) 2019-12-22T22:18:01.236 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:01.237 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 36 M 2019-12-22T22:18:01.237 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 127 M 2019-12-22T22:18:01.237 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading packages: 2019-12-22T22:18:01.456 INFO:teuthology.orchestra.run.smithi043.stdout:Resolving Dependencies 2019-12-22T22:18:01.456 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:18:01.456 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package ceph-test.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:01.476 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:01.507 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: jq for package: 2:ceph-test-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:01.507 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: socat for package: 2:ceph-test-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:01.509 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:18:01.509 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package jq.x86_64 0:1.5-1.el7 will be installed 2019-12-22T22:18:01.510 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.5-1.el7.x86_64 2019-12-22T22:18:01.510 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2019-12-22T22:18:01.513 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2019-12-22T22:18:01.515 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:18:01.516 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2019-12-22T22:18:01.895 INFO:teuthology.orchestra.run.smithi043.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:01.935 INFO:teuthology.orchestra.run.smithi091.stdout:-------------------------------------------------------------------------------- 2019-12-22T22:18:01.935 INFO:teuthology.orchestra.run.smithi091.stdout:Total 21 MB/s | 36 MB 00:01 2019-12-22T22:18:01.938 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction check 2019-12-22T22:18:01.956 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction test 2019-12-22T22:18:02.015 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction test succeeded 2019-12-22T22:18:02.015 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction 2019-12-22T22:18:02.121 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:02.122 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies Resolved 2019-12-22T22:18:02.123 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:02.123 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:02.123 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository 2019-12-22T22:18:02.123 INFO:teuthology.orchestra.run.smithi043.stdout: Size 2019-12-22T22:18:02.123 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:02.124 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2019-12-22T22:18:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test x86_64 2:12.2.12-835.ga6ab182.el7 ceph 35 M 2019-12-22T22:18:02.124 INFO:teuthology.orchestra.run.smithi043.stdout:Installing for dependencies: 2019-12-22T22:18:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: jq x86_64 1.5-1.el7 epel 153 k 2019-12-22T22:18:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2019-12-22T22:18:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2019-12-22T22:18:02.124 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2019-12-22T22:18:02.125 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:02.125 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2019-12-22T22:18:02.125 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:02.125 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package (+4 Dependent packages) 2019-12-22T22:18:02.125 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:02.125 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 36 M 2019-12-22T22:18:02.126 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 127 M 2019-12-22T22:18:02.126 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading packages: 2019-12-22T22:18:02.712 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2019-12-22T22:18:03.040 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2019-12-22T22:18:03.373 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2019-12-22T22:18:06.068 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2019-12-22T22:18:06.068 INFO:teuthology.orchestra.run.smithi107.stdout:Total 7.4 MB/s | 36 MB 00:04 2019-12-22T22:18:06.070 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2019-12-22T22:18:06.088 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2019-12-22T22:18:06.150 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded 2019-12-22T22:18:06.151 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2019-12-22T22:18:06.883 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2019-12-22T22:18:07.207 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2019-12-22T22:18:07.249 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : jq-1.5-1.el7.x86_64 4/5 2019-12-22T22:18:07.516 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2019-12-22T22:18:07.542 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:ceph-test-12.2.12-835.ga6ab182.el7.x86_64 5/5 2019-12-22T22:18:07.676 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 1/5 2019-12-22T22:18:07.784 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 2/5 2019-12-22T22:18:07.868 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:ceph-test-12.2.12-835.ga6ab182.el7.x86_64 3/5 2019-12-22T22:18:08.002 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : jq-1.5-1.el7.x86_64 4/5 2019-12-22T22:18:08.730 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 5/5 2019-12-22T22:18:08.730 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:08.730 INFO:teuthology.orchestra.run.smithi091.stdout:Installed: 2019-12-22T22:18:08.731 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-test.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:08.731 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:08.731 INFO:teuthology.orchestra.run.smithi091.stdout:Dependency Installed: 2019-12-22T22:18:08.731 INFO:teuthology.orchestra.run.smithi091.stdout: jq.x86_64 0:1.5-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2019-12-22T22:18:08.732 INFO:teuthology.orchestra.run.smithi091.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2019-12-22T22:18:08.732 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:08.732 INFO:teuthology.orchestra.run.smithi091.stdout:Complete! 2019-12-22T22:18:08.854 INFO:teuthology.orchestra.run.smithi091:> sudo yum -y install ceph 2019-12-22T22:18:08.884 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2019-12-22T22:18:08.884 INFO:teuthology.orchestra.run.smithi043.stdout:Total 5.3 MB/s | 36 MB 00:06 2019-12-22T22:18:08.884 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2019-12-22T22:18:08.903 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2019-12-22T22:18:08.992 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded 2019-12-22T22:18:08.992 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2019-12-22T22:18:09.127 INFO:teuthology.orchestra.run.smithi091.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:09.173 INFO:teuthology.orchestra.run.smithi091.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:09.174 INFO:teuthology.orchestra.run.smithi091.stdout: * base: mirror.umd.edu 2019-12-22T22:18:09.174 INFO:teuthology.orchestra.run.smithi091.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:09.174 INFO:teuthology.orchestra.run.smithi091.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:09.175 INFO:teuthology.orchestra.run.smithi091.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:09.828 INFO:teuthology.orchestra.run.smithi091.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:10.105 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2019-12-22T22:18:10.165 INFO:teuthology.orchestra.run.smithi091.stdout:Resolving Dependencies 2019-12-22T22:18:10.166 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:18:10.166 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package ceph.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:10.170 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: ceph-mds = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:10.198 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: ceph-mgr = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:10.199 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: ceph-osd = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:10.200 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: ceph-mon = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:10.202 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:18:10.202 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package ceph-mds.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:10.213 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package ceph-mgr.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:10.215 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:10.217 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:10.217 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-jinja2 for package: 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:10.218 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:10.219 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:10.220 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package ceph-mon.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:10.224 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-flask for package: 2:ceph-mon-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:10.225 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package ceph-osd.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:10.227 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:18:10.227 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2019-12-22T22:18:10.228 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2019-12-22T22:18:10.229 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-flask.noarch 1:0.10.1-4.el7 will be installed 2019-12-22T22:18:10.229 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-itsdangerous for package: 1:python-flask-0.10.1-4.el7.noarch 2019-12-22T22:18:10.230 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-jinja2.noarch 0:2.7.2-4.el7 will be installed 2019-12-22T22:18:10.230 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-babel >= 0.8 for package: python-jinja2-2.7.2-4.el7.noarch 2019-12-22T22:18:10.232 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-markupsafe for package: python-jinja2-2.7.2-4.el7.noarch 2019-12-22T22:18:10.232 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2019-12-22T22:18:10.233 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-22T22:18:10.234 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-22T22:18:10.234 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-22T22:18:10.235 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-22T22:18:10.236 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2019-12-22T22:18:10.237 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2019-12-22T22:18:10.237 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:18:10.237 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-babel.noarch 0:0.9.6-8.el7 will be installed 2019-12-22T22:18:10.237 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-itsdangerous.noarch 0:0.23-2.el7 will be installed 2019-12-22T22:18:10.238 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2019-12-22T22:18:10.238 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2019-12-22T22:18:10.239 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2019-12-22T22:18:10.239 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2019-12-22T22:18:10.239 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2019-12-22T22:18:10.239 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2019-12-22T22:18:10.239 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2019-12-22T22:18:10.240 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:18:10.240 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2019-12-22T22:18:10.240 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2019-12-22T22:18:10.241 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:18:10.241 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2019-12-22T22:18:10.241 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2019-12-22T22:18:10.241 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:18:10.241 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2019-12-22T22:18:10.433 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2019-12-22T22:18:10.771 INFO:teuthology.orchestra.run.smithi091.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:10.809 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2019-12-22T22:18:10.818 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:10.819 INFO:teuthology.orchestra.run.smithi091.stdout:Dependencies Resolved 2019-12-22T22:18:10.821 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:10.821 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:10.822 INFO:teuthology.orchestra.run.smithi091.stdout: Package Arch Version Repository 2019-12-22T22:18:10.822 INFO:teuthology.orchestra.run.smithi091.stdout: Size 2019-12-22T22:18:10.822 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:10.822 INFO:teuthology.orchestra.run.smithi091.stdout:Installing: 2019-12-22T22:18:10.823 INFO:teuthology.orchestra.run.smithi091.stdout: ceph x86_64 2:12.2.12-835.ga6ab182.el7 ceph 2.0 k 2019-12-22T22:18:10.823 INFO:teuthology.orchestra.run.smithi091.stdout:Installing for dependencies: 2019-12-22T22:18:10.823 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mds x86_64 2:12.2.12-835.ga6ab182.el7 ceph 3.7 M 2019-12-22T22:18:10.823 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr x86_64 2:12.2.12-835.ga6ab182.el7 ceph 3.7 M 2019-12-22T22:18:10.823 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mon x86_64 2:12.2.12-835.ga6ab182.el7 ceph 5.2 M 2019-12-22T22:18:10.823 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-osd x86_64 2:12.2.12-835.ga6ab182.el7 ceph 13 M 2019-12-22T22:18:10.824 INFO:teuthology.orchestra.run.smithi091.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2019-12-22T22:18:10.824 INFO:teuthology.orchestra.run.smithi091.stdout: python-babel noarch 0.9.6-8.el7 base 1.4 M 2019-12-22T22:18:10.824 INFO:teuthology.orchestra.run.smithi091.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2019-12-22T22:18:10.824 INFO:teuthology.orchestra.run.smithi091.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2019-12-22T22:18:10.825 INFO:teuthology.orchestra.run.smithi091.stdout: python-flask noarch 1:0.10.1-4.el7 extras 204 k 2019-12-22T22:18:10.825 INFO:teuthology.orchestra.run.smithi091.stdout: python-itsdangerous noarch 0.23-2.el7 extras 24 k 2019-12-22T22:18:10.825 INFO:teuthology.orchestra.run.smithi091.stdout: python-jinja2 noarch 2.7.2-4.el7 base 519 k 2019-12-22T22:18:10.825 INFO:teuthology.orchestra.run.smithi091.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2019-12-22T22:18:10.825 INFO:teuthology.orchestra.run.smithi091.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2019-12-22T22:18:10.825 INFO:teuthology.orchestra.run.smithi091.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2019-12-22T22:18:10.826 INFO:teuthology.orchestra.run.smithi091.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2019-12-22T22:18:10.826 INFO:teuthology.orchestra.run.smithi091.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2019-12-22T22:18:10.826 INFO:teuthology.orchestra.run.smithi091.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2019-12-22T22:18:10.826 INFO:teuthology.orchestra.run.smithi091.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2019-12-22T22:18:10.827 INFO:teuthology.orchestra.run.smithi091.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2019-12-22T22:18:10.827 INFO:teuthology.orchestra.run.smithi091.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2019-12-22T22:18:10.827 INFO:teuthology.orchestra.run.smithi091.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2019-12-22T22:18:10.827 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:10.827 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction Summary 2019-12-22T22:18:10.828 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:10.828 INFO:teuthology.orchestra.run.smithi091.stdout:Install 1 Package (+21 Dependent packages) 2019-12-22T22:18:10.828 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:10.829 INFO:teuthology.orchestra.run.smithi091.stdout:Total download size: 30 M 2019-12-22T22:18:10.829 INFO:teuthology.orchestra.run.smithi091.stdout:Installed size: 106 M 2019-12-22T22:18:10.829 INFO:teuthology.orchestra.run.smithi091.stdout:Downloading packages: 2019-12-22T22:18:11.312 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : jq-1.5-1.el7.x86_64 4/5 2019-12-22T22:18:11.632 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:ceph-test-12.2.12-835.ga6ab182.el7.x86_64 5/5 2019-12-22T22:18:11.758 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 1/5 2019-12-22T22:18:11.883 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 2/5 2019-12-22T22:18:11.983 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:ceph-test-12.2.12-835.ga6ab182.el7.x86_64 3/5 2019-12-22T22:18:12.219 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : jq-1.5-1.el7.x86_64 4/5 2019-12-22T22:18:12.679 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 5/5 2019-12-22T22:18:12.679 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:12.679 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2019-12-22T22:18:12.679 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:12.679 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:12.680 INFO:teuthology.orchestra.run.smithi107.stdout:Dependency Installed: 2019-12-22T22:18:12.680 INFO:teuthology.orchestra.run.smithi107.stdout: jq.x86_64 0:1.5-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2019-12-22T22:18:12.680 INFO:teuthology.orchestra.run.smithi107.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2019-12-22T22:18:12.680 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:12.680 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2019-12-22T22:18:12.811 INFO:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph 2019-12-22T22:18:12.918 INFO:teuthology.orchestra.run.smithi091.stdout:-------------------------------------------------------------------------------- 2019-12-22T22:18:12.919 INFO:teuthology.orchestra.run.smithi091.stdout:Total 15 MB/s | 30 MB 00:02 2019-12-22T22:18:12.949 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction check 2019-12-22T22:18:12.988 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction test 2019-12-22T22:18:13.057 INFO:teuthology.orchestra.run.smithi107.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:13.092 INFO:teuthology.orchestra.run.smithi107.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:13.103 INFO:teuthology.orchestra.run.smithi107.stdout: * base: mirror.umd.edu 2019-12-22T22:18:13.103 INFO:teuthology.orchestra.run.smithi107.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:13.104 INFO:teuthology.orchestra.run.smithi107.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:13.104 INFO:teuthology.orchestra.run.smithi107.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:13.268 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction test succeeded 2019-12-22T22:18:13.269 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction 2019-12-22T22:18:13.740 INFO:teuthology.orchestra.run.smithi107.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:14.068 INFO:teuthology.orchestra.run.smithi107.stdout:Resolving Dependencies 2019-12-22T22:18:14.069 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:18:14.069 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package ceph.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:14.071 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: ceph-mds = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:14.101 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: ceph-mgr = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:14.102 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: ceph-osd = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:14.103 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: ceph-mon = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:14.104 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:18:14.105 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package ceph-mds.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:14.116 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package ceph-mgr.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:14.118 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:14.119 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:14.120 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-jinja2 for package: 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:14.121 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:14.122 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:14.123 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package ceph-mon.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:14.127 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-flask for package: 2:ceph-mon-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:14.127 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package ceph-osd.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:14.141 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:18:14.200 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2019-12-22T22:18:14.200 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2019-12-22T22:18:14.200 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-flask.noarch 1:0.10.1-4.el7 will be installed 2019-12-22T22:18:14.200 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-itsdangerous for package: 1:python-flask-0.10.1-4.el7.noarch 2019-12-22T22:18:14.200 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-jinja2.noarch 0:2.7.2-4.el7 will be installed 2019-12-22T22:18:14.201 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-babel >= 0.8 for package: python-jinja2-2.7.2-4.el7.noarch 2019-12-22T22:18:14.201 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-markupsafe for package: python-jinja2-2.7.2-4.el7.noarch 2019-12-22T22:18:14.201 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2019-12-22T22:18:14.201 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-22T22:18:14.201 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-22T22:18:14.201 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-22T22:18:14.202 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-22T22:18:14.206 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2019-12-22T22:18:14.206 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2019-12-22T22:18:14.206 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:18:14.207 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-babel.noarch 0:0.9.6-8.el7 will be installed 2019-12-22T22:18:14.207 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-itsdangerous.noarch 0:0.23-2.el7 will be installed 2019-12-22T22:18:14.207 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2019-12-22T22:18:14.207 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2019-12-22T22:18:14.207 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2019-12-22T22:18:14.207 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2019-12-22T22:18:14.207 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2019-12-22T22:18:14.208 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2019-12-22T22:18:14.208 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2019-12-22T22:18:14.208 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:18:14.208 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2019-12-22T22:18:14.208 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2019-12-22T22:18:14.208 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:18:14.209 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2019-12-22T22:18:14.209 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2019-12-22T22:18:14.209 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:18:14.209 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2019-12-22T22:18:14.505 INFO:teuthology.orchestra.run.smithi107.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:14.514 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 1/22 2019-12-22T22:18:14.722 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:14.722 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies Resolved 2019-12-22T22:18:14.725 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:14.725 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:14.725 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository 2019-12-22T22:18:14.725 INFO:teuthology.orchestra.run.smithi107.stdout: Size 2019-12-22T22:18:14.725 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:14.726 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2019-12-22T22:18:14.726 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:12.2.12-835.ga6ab182.el7 ceph 2.0 k 2019-12-22T22:18:14.726 INFO:teuthology.orchestra.run.smithi107.stdout:Installing for dependencies: 2019-12-22T22:18:14.726 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:12.2.12-835.ga6ab182.el7 ceph 3.7 M 2019-12-22T22:18:14.726 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:12.2.12-835.ga6ab182.el7 ceph 3.7 M 2019-12-22T22:18:14.726 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:12.2.12-835.ga6ab182.el7 ceph 5.2 M 2019-12-22T22:18:14.726 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:12.2.12-835.ga6ab182.el7 ceph 13 M 2019-12-22T22:18:14.727 INFO:teuthology.orchestra.run.smithi107.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2019-12-22T22:18:14.727 INFO:teuthology.orchestra.run.smithi107.stdout: python-babel noarch 0.9.6-8.el7 base 1.4 M 2019-12-22T22:18:14.727 INFO:teuthology.orchestra.run.smithi107.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2019-12-22T22:18:14.727 INFO:teuthology.orchestra.run.smithi107.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2019-12-22T22:18:14.727 INFO:teuthology.orchestra.run.smithi107.stdout: python-flask noarch 1:0.10.1-4.el7 extras 204 k 2019-12-22T22:18:14.727 INFO:teuthology.orchestra.run.smithi107.stdout: python-itsdangerous noarch 0.23-2.el7 extras 24 k 2019-12-22T22:18:14.727 INFO:teuthology.orchestra.run.smithi107.stdout: python-jinja2 noarch 2.7.2-4.el7 base 519 k 2019-12-22T22:18:14.728 INFO:teuthology.orchestra.run.smithi107.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2019-12-22T22:18:14.728 INFO:teuthology.orchestra.run.smithi107.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2019-12-22T22:18:14.728 INFO:teuthology.orchestra.run.smithi107.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2019-12-22T22:18:14.728 INFO:teuthology.orchestra.run.smithi107.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2019-12-22T22:18:14.728 INFO:teuthology.orchestra.run.smithi107.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2019-12-22T22:18:14.728 INFO:teuthology.orchestra.run.smithi107.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2019-12-22T22:18:14.728 INFO:teuthology.orchestra.run.smithi107.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2019-12-22T22:18:14.729 INFO:teuthology.orchestra.run.smithi107.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2019-12-22T22:18:14.729 INFO:teuthology.orchestra.run.smithi107.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2019-12-22T22:18:14.729 INFO:teuthology.orchestra.run.smithi107.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2019-12-22T22:18:14.729 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:14.729 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2019-12-22T22:18:14.729 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:14.729 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package (+21 Dependent packages) 2019-12-22T22:18:14.729 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:14.730 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 30 M 2019-12-22T22:18:14.730 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 106 M 2019-12-22T22:18:14.731 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading packages: 2019-12-22T22:18:14.791 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python-webob-1.2.3-7.el7.noarch 2/22 2019-12-22T22:18:14.836 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : jq-1.5-1.el7.x86_64 4/5 2019-12-22T22:18:15.096 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:ceph-test-12.2.12-835.ga6ab182.el7.x86_64 5/5 2019-12-22T22:18:15.211 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 1/5 2019-12-22T22:18:15.231 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 3/22 2019-12-22T22:18:15.320 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 2/5 2019-12-22T22:18:15.404 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:ceph-test-12.2.12-835.ga6ab182.el7.x86_64 3/5 2019-12-22T22:18:15.513 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : jq-1.5-1.el7.x86_64 4/5 2019-12-22T22:18:15.589 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 4/22 2019-12-22T22:18:15.915 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 5/22 2019-12-22T22:18:16.014 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 5/5 2019-12-22T22:18:16.014 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:16.015 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2019-12-22T22:18:16.015 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:16.015 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:16.015 INFO:teuthology.orchestra.run.smithi043.stdout:Dependency Installed: 2019-12-22T22:18:16.015 INFO:teuthology.orchestra.run.smithi043.stdout: jq.x86_64 0:1.5-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2019-12-22T22:18:16.015 INFO:teuthology.orchestra.run.smithi043.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2019-12-22T22:18:16.016 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:16.016 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2019-12-22T22:18:16.134 INFO:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph 2019-12-22T22:18:16.366 INFO:teuthology.orchestra.run.smithi043.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:16.407 INFO:teuthology.orchestra.run.smithi043.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:16.408 INFO:teuthology.orchestra.run.smithi043.stdout: * base: mirror.umd.edu 2019-12-22T22:18:16.408 INFO:teuthology.orchestra.run.smithi043.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:16.409 INFO:teuthology.orchestra.run.smithi043.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:16.409 INFO:teuthology.orchestra.run.smithi043.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:16.457 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 6/22 2019-12-22T22:18:16.658 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2019-12-22T22:18:16.659 INFO:teuthology.orchestra.run.smithi107.stdout:Total 16 MB/s | 30 MB 00:01 2019-12-22T22:18:16.690 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2019-12-22T22:18:16.733 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2019-12-22T22:18:16.829 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 7/22 2019-12-22T22:18:17.034 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded 2019-12-22T22:18:17.034 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2019-12-22T22:18:17.082 INFO:teuthology.orchestra.run.smithi043.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:17.301 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 8/22 2019-12-22T22:18:17.426 INFO:teuthology.orchestra.run.smithi043.stdout:Resolving Dependencies 2019-12-22T22:18:17.427 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:18:17.427 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package ceph.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:17.429 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: ceph-mds = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:17.460 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: ceph-mgr = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:17.462 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: ceph-osd = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:17.463 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: ceph-mon = 2:12.2.12-835.ga6ab182.el7 for package: 2:ceph-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:17.464 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:18:17.465 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package ceph-mds.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:17.478 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package ceph-mgr.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:17.479 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:17.480 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:17.481 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-jinja2 for package: 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:17.482 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:17.483 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:17.484 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package ceph-mon.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:17.489 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-flask for package: 2:ceph-mon-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:17.489 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package ceph-osd.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:17.492 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:18:17.493 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2019-12-22T22:18:17.493 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2019-12-22T22:18:17.494 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-flask.noarch 1:0.10.1-4.el7 will be installed 2019-12-22T22:18:17.495 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-itsdangerous for package: 1:python-flask-0.10.1-4.el7.noarch 2019-12-22T22:18:17.495 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-jinja2.noarch 0:2.7.2-4.el7 will be installed 2019-12-22T22:18:17.496 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-babel >= 0.8 for package: python-jinja2-2.7.2-4.el7.noarch 2019-12-22T22:18:17.497 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-markupsafe for package: python-jinja2-2.7.2-4.el7.noarch 2019-12-22T22:18:17.497 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2019-12-22T22:18:17.498 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-22T22:18:17.499 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-22T22:18:17.500 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-22T22:18:17.501 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-22T22:18:17.502 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2019-12-22T22:18:17.503 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2019-12-22T22:18:17.503 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:18:17.504 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-babel.noarch 0:0.9.6-8.el7 will be installed 2019-12-22T22:18:17.504 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-itsdangerous.noarch 0:0.23-2.el7 will be installed 2019-12-22T22:18:17.504 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2019-12-22T22:18:17.505 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2019-12-22T22:18:17.505 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2019-12-22T22:18:17.506 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2019-12-22T22:18:17.506 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2019-12-22T22:18:17.506 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2019-12-22T22:18:17.506 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2019-12-22T22:18:17.506 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:18:17.506 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2019-12-22T22:18:17.506 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2019-12-22T22:18:17.507 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:18:17.507 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2019-12-22T22:18:17.507 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2019-12-22T22:18:17.508 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:18:17.508 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2019-12-22T22:18:17.805 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python-mako-0.8.1-2.el7.noarch 9/22 2019-12-22T22:18:17.853 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 1/22 2019-12-22T22:18:17.888 INFO:teuthology.orchestra.run.smithi043.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:18.118 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:18.118 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies Resolved 2019-12-22T22:18:18.123 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:18.123 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:18.123 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository 2019-12-22T22:18:18.123 INFO:teuthology.orchestra.run.smithi043.stdout: Size 2019-12-22T22:18:18.123 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:18.123 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2019-12-22T22:18:18.124 INFO:teuthology.orchestra.run.smithi043.stdout: ceph x86_64 2:12.2.12-835.ga6ab182.el7 ceph 2.0 k 2019-12-22T22:18:18.124 INFO:teuthology.orchestra.run.smithi043.stdout:Installing for dependencies: 2019-12-22T22:18:18.124 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds x86_64 2:12.2.12-835.ga6ab182.el7 ceph 3.7 M 2019-12-22T22:18:18.124 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr x86_64 2:12.2.12-835.ga6ab182.el7 ceph 3.7 M 2019-12-22T22:18:18.124 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon x86_64 2:12.2.12-835.ga6ab182.el7 ceph 5.2 M 2019-12-22T22:18:18.124 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd x86_64 2:12.2.12-835.ga6ab182.el7 ceph 13 M 2019-12-22T22:18:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2019-12-22T22:18:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: python-babel noarch 0.9.6-8.el7 base 1.4 M 2019-12-22T22:18:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2019-12-22T22:18:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2019-12-22T22:18:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: python-flask noarch 1:0.10.1-4.el7 extras 204 k 2019-12-22T22:18:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: python-itsdangerous noarch 0.23-2.el7 extras 24 k 2019-12-22T22:18:18.125 INFO:teuthology.orchestra.run.smithi043.stdout: python-jinja2 noarch 2.7.2-4.el7 base 519 k 2019-12-22T22:18:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2019-12-22T22:18:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2019-12-22T22:18:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2019-12-22T22:18:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2019-12-22T22:18:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2019-12-22T22:18:18.126 INFO:teuthology.orchestra.run.smithi043.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2019-12-22T22:18:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2019-12-22T22:18:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2019-12-22T22:18:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2019-12-22T22:18:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2019-12-22T22:18:18.127 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:18.127 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2019-12-22T22:18:18.128 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:18.128 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package (+21 Dependent packages) 2019-12-22T22:18:18.128 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:18.128 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 30 M 2019-12-22T22:18:18.128 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 106 M 2019-12-22T22:18:18.129 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading packages: 2019-12-22T22:18:18.600 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python-webob-1.2.3-7.el7.noarch 2/22 2019-12-22T22:18:19.072 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 3/22 2019-12-22T22:18:19.434 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 10/22 2019-12-22T22:18:19.440 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 4/22 2019-12-22T22:18:19.741 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 5/22 2019-12-22T22:18:19.892 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:ceph-osd-12.2.12-835.ga6ab182.el7.x86_64 11/22 2019-12-22T22:18:20.226 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 6/22 2019-12-22T22:18:20.523 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 12/22 2019-12-22T22:18:20.575 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2019-12-22T22:18:20.575 INFO:teuthology.orchestra.run.smithi043.stdout:Total 12 MB/s | 30 MB 00:02 2019-12-22T22:18:20.608 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2019-12-22T22:18:20.618 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 7/22 2019-12-22T22:18:20.653 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2019-12-22T22:18:20.925 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded 2019-12-22T22:18:20.925 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2019-12-22T22:18:21.075 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python-babel-0.9.6-8.el7.noarch 13/22 2019-12-22T22:18:21.087 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 8/22 2019-12-22T22:18:21.435 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python-jinja2-2.7.2-4.el7.noarch 14/22 2019-12-22T22:18:21.571 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python-mako-0.8.1-2.el7.noarch 9/22 2019-12-22T22:18:21.687 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 1/22 2019-12-22T22:18:21.874 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python-itsdangerous-0.23-2.el7.noarch 15/22 2019-12-22T22:18:22.111 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python-webob-1.2.3-7.el7.noarch 2/22 2019-12-22T22:18:22.418 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 3/22 2019-12-22T22:18:22.788 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 4/22 2019-12-22T22:18:22.797 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 1:python-flask-0.10.1-4.el7.noarch 16/22 2019-12-22T22:18:23.098 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 5/22 2019-12-22T22:18:23.202 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 10/22 2019-12-22T22:18:23.313 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:ceph-mon-12.2.12-835.ga6ab182.el7.x86_64 17/22 2019-12-22T22:18:23.591 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 6/22 2019-12-22T22:18:23.642 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 18/22 2019-12-22T22:18:23.692 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:ceph-osd-12.2.12-835.ga6ab182.el7.x86_64 11/22 2019-12-22T22:18:23.957 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 7/22 2019-12-22T22:18:24.375 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 12/22 2019-12-22T22:18:24.405 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 8/22 2019-12-22T22:18:24.493 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 19/22 2019-12-22T22:18:24.897 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python-mako-0.8.1-2.el7.noarch 9/22 2019-12-22T22:18:24.951 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python-babel-0.9.6-8.el7.noarch 13/22 2019-12-22T22:18:25.293 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python-jinja2-2.7.2-4.el7.noarch 14/22 2019-12-22T22:18:25.351 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 20/22 2019-12-22T22:18:25.729 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python-itsdangerous-0.23-2.el7.noarch 15/22 2019-12-22T22:18:25.735 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:ceph-mds-12.2.12-835.ga6ab182.el7.x86_64 21/22 2019-12-22T22:18:25.978 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:ceph-12.2.12-835.ga6ab182.el7.x86_64 22/22 2019-12-22T22:18:26.097 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:ceph-mds-12.2.12-835.ga6ab182.el7.x86_64 1/22 2019-12-22T22:18:26.197 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 1:python-flask-0.10.1-4.el7.noarch 2/22 2019-12-22T22:18:26.289 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 3/22 2019-12-22T22:18:26.423 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 4/22 2019-12-22T22:18:26.540 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 5/22 2019-12-22T22:18:26.546 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 10/22 2019-12-22T22:18:26.630 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 1:python-flask-0.10.1-4.el7.noarch 16/22 2019-12-22T22:18:26.682 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python-itsdangerous-0.23-2.el7.noarch 6/22 2019-12-22T22:18:26.808 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python-jinja2-2.7.2-4.el7.noarch 7/22 2019-12-22T22:18:26.937 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python-babel-0.9.6-8.el7.noarch 8/22 2019-12-22T22:18:26.980 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:ceph-osd-12.2.12-835.ga6ab182.el7.x86_64 11/22 2019-12-22T22:18:27.076 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:ceph-mon-12.2.12-835.ga6ab182.el7.x86_64 17/22 2019-12-22T22:18:27.084 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 9/22 2019-12-22T22:18:27.201 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 10/22 2019-12-22T22:18:27.376 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 11/22 2019-12-22T22:18:27.503 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:ceph-osd-12.2.12-835.ga6ab182.el7.x86_64 12/22 2019-12-22T22:18:27.543 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 18/22 2019-12-22T22:18:27.619 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 13/22 2019-12-22T22:18:27.661 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 12/22 2019-12-22T22:18:27.752 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 14/22 2019-12-22T22:18:27.870 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 15/22 2019-12-22T22:18:27.988 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:ceph-12.2.12-835.ga6ab182.el7.x86_64 16/22 2019-12-22T22:18:28.107 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 17/22 2019-12-22T22:18:28.169 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python-babel-0.9.6-8.el7.noarch 13/22 2019-12-22T22:18:28.188 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 18/22 2019-12-22T22:18:28.301 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 19/22 2019-12-22T22:18:28.387 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 19/22 2019-12-22T22:18:28.389 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:ceph-mon-12.2.12-835.ga6ab182.el7.x86_64 20/22 2019-12-22T22:18:28.463 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python-jinja2-2.7.2-4.el7.noarch 14/22 2019-12-22T22:18:28.464 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 21/22 2019-12-22T22:18:28.912 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python-itsdangerous-0.23-2.el7.noarch 15/22 2019-12-22T22:18:29.202 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 22/22 2019-12-22T22:18:29.202 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:29.202 INFO:teuthology.orchestra.run.smithi091.stdout:Installed: 2019-12-22T22:18:29.202 INFO:teuthology.orchestra.run.smithi091.stdout: ceph.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:29.203 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:29.203 INFO:teuthology.orchestra.run.smithi091.stdout:Dependency Installed: 2019-12-22T22:18:29.203 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mds.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:29.203 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mgr.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:29.204 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-mon.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:29.204 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-osd.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:29.204 INFO:teuthology.orchestra.run.smithi091.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2019-12-22T22:18:29.204 INFO:teuthology.orchestra.run.smithi091.stdout: python-babel.noarch 0:0.9.6-8.el7 2019-12-22T22:18:29.204 INFO:teuthology.orchestra.run.smithi091.stdout: python-beaker.noarch 0:1.5.4-10.el7 2019-12-22T22:18:29.205 INFO:teuthology.orchestra.run.smithi091.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2019-12-22T22:18:29.205 INFO:teuthology.orchestra.run.smithi091.stdout: python-flask.noarch 1:0.10.1-4.el7 2019-12-22T22:18:29.205 INFO:teuthology.orchestra.run.smithi091.stdout: python-itsdangerous.noarch 0:0.23-2.el7 2019-12-22T22:18:29.205 INFO:teuthology.orchestra.run.smithi091.stdout: python-jinja2.noarch 0:2.7.2-4.el7 2019-12-22T22:18:29.206 INFO:teuthology.orchestra.run.smithi091.stdout: python-mako.noarch 0:0.8.1-2.el7 2019-12-22T22:18:29.206 INFO:teuthology.orchestra.run.smithi091.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2019-12-22T22:18:29.206 INFO:teuthology.orchestra.run.smithi091.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2019-12-22T22:18:29.206 INFO:teuthology.orchestra.run.smithi091.stdout: python-pecan.noarch 0:0.4.5-2.el7 2019-12-22T22:18:29.206 INFO:teuthology.orchestra.run.smithi091.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2019-12-22T22:18:29.207 INFO:teuthology.orchestra.run.smithi091.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2019-12-22T22:18:29.207 INFO:teuthology.orchestra.run.smithi091.stdout: python-tempita.noarch 0:0.5.1-6.el7 2019-12-22T22:18:29.207 INFO:teuthology.orchestra.run.smithi091.stdout: python-webob.noarch 0:1.2.3-7.el7 2019-12-22T22:18:29.207 INFO:teuthology.orchestra.run.smithi091.stdout: python-webtest.noarch 0:1.3.4-6.el7 2019-12-22T22:18:29.208 INFO:teuthology.orchestra.run.smithi091.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2019-12-22T22:18:29.208 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:29.208 INFO:teuthology.orchestra.run.smithi091.stdout:Complete! 2019-12-22T22:18:29.293 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 20/22 2019-12-22T22:18:29.337 INFO:teuthology.orchestra.run.smithi091:> sudo yum -y install ceph-mgr 2019-12-22T22:18:29.576 INFO:teuthology.orchestra.run.smithi091.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:29.614 INFO:teuthology.orchestra.run.smithi091.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:29.615 INFO:teuthology.orchestra.run.smithi091.stdout: * base: mirror.umd.edu 2019-12-22T22:18:29.616 INFO:teuthology.orchestra.run.smithi091.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:29.616 INFO:teuthology.orchestra.run.smithi091.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:29.616 INFO:teuthology.orchestra.run.smithi091.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:29.665 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:ceph-mds-12.2.12-835.ga6ab182.el7.x86_64 21/22 2019-12-22T22:18:29.824 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 1:python-flask-0.10.1-4.el7.noarch 16/22 2019-12-22T22:18:29.935 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:ceph-12.2.12-835.ga6ab182.el7.x86_64 22/22 2019-12-22T22:18:30.038 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:ceph-mds-12.2.12-835.ga6ab182.el7.x86_64 1/22 2019-12-22T22:18:30.197 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 1:python-flask-0.10.1-4.el7.noarch 2/22 2019-12-22T22:18:30.204 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:ceph-mon-12.2.12-835.ga6ab182.el7.x86_64 17/22 2019-12-22T22:18:30.261 INFO:teuthology.orchestra.run.smithi091.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:30.297 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 3/22 2019-12-22T22:18:30.456 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 4/22 2019-12-22T22:18:30.573 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 5/22 2019-12-22T22:18:30.587 INFO:teuthology.orchestra.run.smithi091.stdout:Package 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:18:30.588 INFO:teuthology.orchestra.run.smithi091.stdout:Nothing to do 2019-12-22T22:18:30.606 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 18/22 2019-12-22T22:18:30.648 INFO:teuthology.orchestra.run.smithi091:> sudo yum -y install ceph-fuse 2019-12-22T22:18:30.726 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python-itsdangerous-0.23-2.el7.noarch 6/22 2019-12-22T22:18:30.837 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python-jinja2-2.7.2-4.el7.noarch 7/22 2019-12-22T22:18:30.859 INFO:teuthology.orchestra.run.smithi091.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:30.899 INFO:teuthology.orchestra.run.smithi091.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:30.900 INFO:teuthology.orchestra.run.smithi091.stdout: * base: mirror.umd.edu 2019-12-22T22:18:30.901 INFO:teuthology.orchestra.run.smithi091.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:30.901 INFO:teuthology.orchestra.run.smithi091.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:30.901 INFO:teuthology.orchestra.run.smithi091.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:30.975 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python-babel-0.9.6-8.el7.noarch 8/22 2019-12-22T22:18:31.092 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 9/22 2019-12-22T22:18:31.229 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 10/22 2019-12-22T22:18:31.318 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 11/22 2019-12-22T22:18:31.429 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 19/22 2019-12-22T22:18:31.443 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:ceph-osd-12.2.12-835.ga6ab182.el7.x86_64 12/22 2019-12-22T22:18:31.544 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 13/22 2019-12-22T22:18:31.550 INFO:teuthology.orchestra.run.smithi091.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:31.649 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 14/22 2019-12-22T22:18:31.778 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 15/22 2019-12-22T22:18:31.875 INFO:teuthology.orchestra.run.smithi091.stdout:Resolving Dependencies 2019-12-22T22:18:31.876 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:18:31.876 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package ceph-fuse.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:31.887 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:ceph-12.2.12-835.ga6ab182.el7.x86_64 16/22 2019-12-22T22:18:32.062 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 17/22 2019-12-22T22:18:32.183 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 18/22 2019-12-22T22:18:32.257 INFO:teuthology.orchestra.run.smithi091.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:32.305 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 19/22 2019-12-22T22:18:32.330 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 20/22 2019-12-22T22:18:32.414 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:ceph-mon-12.2.12-835.ga6ab182.el7.x86_64 20/22 2019-12-22T22:18:32.425 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:32.425 INFO:teuthology.orchestra.run.smithi091.stdout:Dependencies Resolved 2019-12-22T22:18:32.428 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:32.428 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:32.434 INFO:teuthology.orchestra.run.smithi091.stdout: Package Arch Version Repository Size 2019-12-22T22:18:32.434 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:32.434 INFO:teuthology.orchestra.run.smithi091.stdout:Installing: 2019-12-22T22:18:32.434 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-fuse x86_64 2:12.2.12-835.ga6ab182.el7 ceph 2.5 M 2019-12-22T22:18:32.434 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:32.434 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction Summary 2019-12-22T22:18:32.434 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:32.435 INFO:teuthology.orchestra.run.smithi091.stdout:Install 1 Package 2019-12-22T22:18:32.435 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:32.435 INFO:teuthology.orchestra.run.smithi091.stdout:Total download size: 2.5 M 2019-12-22T22:18:32.435 INFO:teuthology.orchestra.run.smithi091.stdout:Installed size: 8.7 M 2019-12-22T22:18:32.435 INFO:teuthology.orchestra.run.smithi091.stdout:Downloading packages: 2019-12-22T22:18:32.556 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 21/22 2019-12-22T22:18:32.683 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:ceph-mds-12.2.12-835.ga6ab182.el7.x86_64 21/22 2019-12-22T22:18:32.911 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction check 2019-12-22T22:18:32.926 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:ceph-12.2.12-835.ga6ab182.el7.x86_64 22/22 2019-12-22T22:18:32.937 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction test 2019-12-22T22:18:32.938 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction test succeeded 2019-12-22T22:18:32.939 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction 2019-12-22T22:18:33.022 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:ceph-mds-12.2.12-835.ga6ab182.el7.x86_64 1/22 2019-12-22T22:18:33.066 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 22/22 2019-12-22T22:18:33.067 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:33.067 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2019-12-22T22:18:33.067 INFO:teuthology.orchestra.run.smithi107.stdout: ceph.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:33.067 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:33.068 INFO:teuthology.orchestra.run.smithi107.stdout:Dependency Installed: 2019-12-22T22:18:33.068 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2019-12-22T22:18:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: python-babel.noarch 0:0.9.6-8.el7 2019-12-22T22:18:33.069 INFO:teuthology.orchestra.run.smithi107.stdout: python-beaker.noarch 0:1.5.4-10.el7 2019-12-22T22:18:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2019-12-22T22:18:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: python-flask.noarch 1:0.10.1-4.el7 2019-12-22T22:18:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: python-itsdangerous.noarch 0:0.23-2.el7 2019-12-22T22:18:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: python-jinja2.noarch 0:2.7.2-4.el7 2019-12-22T22:18:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: python-mako.noarch 0:0.8.1-2.el7 2019-12-22T22:18:33.070 INFO:teuthology.orchestra.run.smithi107.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2019-12-22T22:18:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2019-12-22T22:18:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: python-pecan.noarch 0:0.4.5-2.el7 2019-12-22T22:18:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2019-12-22T22:18:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2019-12-22T22:18:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: python-tempita.noarch 0:0.5.1-6.el7 2019-12-22T22:18:33.071 INFO:teuthology.orchestra.run.smithi107.stdout: python-webob.noarch 0:1.2.3-7.el7 2019-12-22T22:18:33.072 INFO:teuthology.orchestra.run.smithi107.stdout: python-webtest.noarch 0:1.3.4-6.el7 2019-12-22T22:18:33.072 INFO:teuthology.orchestra.run.smithi107.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2019-12-22T22:18:33.072 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:33.073 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2019-12-22T22:18:33.150 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 1:python-flask-0.10.1-4.el7.noarch 2/22 2019-12-22T22:18:33.225 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 3/22 2019-12-22T22:18:33.326 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 4/22 2019-12-22T22:18:33.418 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 5/22 2019-12-22T22:18:33.479 INFO:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr 2019-12-22T22:18:33.518 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python-itsdangerous-0.23-2.el7.noarch 6/22 2019-12-22T22:18:33.669 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python-jinja2-2.7.2-4.el7.noarch 7/22 2019-12-22T22:18:33.773 INFO:teuthology.orchestra.run.smithi107.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:33.786 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python-babel-0.9.6-8.el7.noarch 8/22 2019-12-22T22:18:33.812 INFO:teuthology.orchestra.run.smithi107.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:33.813 INFO:teuthology.orchestra.run.smithi107.stdout: * base: mirror.umd.edu 2019-12-22T22:18:33.813 INFO:teuthology.orchestra.run.smithi107.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:33.813 INFO:teuthology.orchestra.run.smithi107.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:33.814 INFO:teuthology.orchestra.run.smithi107.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:33.895 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 9/22 2019-12-22T22:18:34.003 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 10/22 2019-12-22T22:18:34.053 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:ceph-fuse-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:34.154 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 11/22 2019-12-22T22:18:34.264 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:ceph-osd-12.2.12-835.ga6ab182.el7.x86_64 12/22 2019-12-22T22:18:34.405 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 13/22 2019-12-22T22:18:34.465 INFO:teuthology.orchestra.run.smithi107.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:34.513 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 14/22 2019-12-22T22:18:34.560 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:ceph-fuse-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:34.560 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:34.561 INFO:teuthology.orchestra.run.smithi091.stdout:Installed: 2019-12-22T22:18:34.561 INFO:teuthology.orchestra.run.smithi091.stdout: ceph-fuse.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:34.561 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:34.561 INFO:teuthology.orchestra.run.smithi091.stdout:Complete! 2019-12-22T22:18:34.664 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 15/22 2019-12-22T22:18:34.671 INFO:teuthology.orchestra.run.smithi091:> sudo yum -y install libcephfs2 2019-12-22T22:18:34.780 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:ceph-12.2.12-835.ga6ab182.el7.x86_64 16/22 2019-12-22T22:18:34.800 INFO:teuthology.orchestra.run.smithi107.stdout:Package 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:18:34.800 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do 2019-12-22T22:18:34.856 INFO:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-fuse 2019-12-22T22:18:34.862 INFO:teuthology.orchestra.run.smithi091.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:34.874 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 17/22 2019-12-22T22:18:34.896 INFO:teuthology.orchestra.run.smithi091.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:34.897 INFO:teuthology.orchestra.run.smithi091.stdout: * base: mirror.umd.edu 2019-12-22T22:18:34.898 INFO:teuthology.orchestra.run.smithi091.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:34.898 INFO:teuthology.orchestra.run.smithi091.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:34.898 INFO:teuthology.orchestra.run.smithi091.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:34.961 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 18/22 2019-12-22T22:18:35.041 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 19/22 2019-12-22T22:18:35.163 INFO:teuthology.orchestra.run.smithi107.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:35.175 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:ceph-mon-12.2.12-835.ga6ab182.el7.x86_64 20/22 2019-12-22T22:18:35.210 INFO:teuthology.orchestra.run.smithi107.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:35.210 INFO:teuthology.orchestra.run.smithi107.stdout: * base: mirror.umd.edu 2019-12-22T22:18:35.211 INFO:teuthology.orchestra.run.smithi107.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:35.211 INFO:teuthology.orchestra.run.smithi107.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:35.211 INFO:teuthology.orchestra.run.smithi107.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:35.269 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 21/22 2019-12-22T22:18:35.529 INFO:teuthology.orchestra.run.smithi091.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:35.743 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 22/22 2019-12-22T22:18:35.744 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:35.744 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2019-12-22T22:18:35.744 INFO:teuthology.orchestra.run.smithi043.stdout: ceph.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:35.744 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:35.744 INFO:teuthology.orchestra.run.smithi043.stdout:Dependency Installed: 2019-12-22T22:18:35.744 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:35.745 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:35.745 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:35.745 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:35.745 INFO:teuthology.orchestra.run.smithi043.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2019-12-22T22:18:35.745 INFO:teuthology.orchestra.run.smithi043.stdout: python-babel.noarch 0:0.9.6-8.el7 2019-12-22T22:18:35.745 INFO:teuthology.orchestra.run.smithi043.stdout: python-beaker.noarch 0:1.5.4-10.el7 2019-12-22T22:18:35.745 INFO:teuthology.orchestra.run.smithi043.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2019-12-22T22:18:35.746 INFO:teuthology.orchestra.run.smithi043.stdout: python-flask.noarch 1:0.10.1-4.el7 2019-12-22T22:18:35.746 INFO:teuthology.orchestra.run.smithi043.stdout: python-itsdangerous.noarch 0:0.23-2.el7 2019-12-22T22:18:35.746 INFO:teuthology.orchestra.run.smithi043.stdout: python-jinja2.noarch 0:2.7.2-4.el7 2019-12-22T22:18:35.746 INFO:teuthology.orchestra.run.smithi043.stdout: python-mako.noarch 0:0.8.1-2.el7 2019-12-22T22:18:35.746 INFO:teuthology.orchestra.run.smithi043.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2019-12-22T22:18:35.746 INFO:teuthology.orchestra.run.smithi043.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2019-12-22T22:18:35.746 INFO:teuthology.orchestra.run.smithi043.stdout: python-pecan.noarch 0:0.4.5-2.el7 2019-12-22T22:18:35.746 INFO:teuthology.orchestra.run.smithi043.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2019-12-22T22:18:35.747 INFO:teuthology.orchestra.run.smithi043.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2019-12-22T22:18:35.747 INFO:teuthology.orchestra.run.smithi043.stdout: python-tempita.noarch 0:0.5.1-6.el7 2019-12-22T22:18:35.747 INFO:teuthology.orchestra.run.smithi043.stdout: python-webob.noarch 0:1.2.3-7.el7 2019-12-22T22:18:35.747 INFO:teuthology.orchestra.run.smithi043.stdout: python-webtest.noarch 0:1.3.4-6.el7 2019-12-22T22:18:35.747 INFO:teuthology.orchestra.run.smithi043.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2019-12-22T22:18:35.747 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:35.747 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2019-12-22T22:18:35.855 INFO:teuthology.orchestra.run.smithi091.stdout:Package 2:libcephfs2-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:18:35.855 INFO:teuthology.orchestra.run.smithi091.stdout:Nothing to do 2019-12-22T22:18:35.868 INFO:teuthology.orchestra.run.smithi107.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:35.869 INFO:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr 2019-12-22T22:18:35.916 INFO:teuthology.orchestra.run.smithi091:> sudo yum -y install libcephfs-devel 2019-12-22T22:18:36.095 INFO:teuthology.orchestra.run.smithi091.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:36.132 INFO:teuthology.orchestra.run.smithi091.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:36.137 INFO:teuthology.orchestra.run.smithi091.stdout: * base: mirror.umd.edu 2019-12-22T22:18:36.137 INFO:teuthology.orchestra.run.smithi091.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:36.137 INFO:teuthology.orchestra.run.smithi091.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:36.137 INFO:teuthology.orchestra.run.smithi091.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:36.145 INFO:teuthology.orchestra.run.smithi043.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:36.188 INFO:teuthology.orchestra.run.smithi043.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:36.189 INFO:teuthology.orchestra.run.smithi043.stdout: * base: mirror.umd.edu 2019-12-22T22:18:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:36.190 INFO:teuthology.orchestra.run.smithi043.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:36.204 INFO:teuthology.orchestra.run.smithi107.stdout:Resolving Dependencies 2019-12-22T22:18:36.205 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:18:36.205 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package ceph-fuse.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:36.576 INFO:teuthology.orchestra.run.smithi107.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:36.721 INFO:teuthology.orchestra.run.smithi091.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:36.754 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:36.754 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies Resolved 2019-12-22T22:18:36.755 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:36.755 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:36.755 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2019-12-22T22:18:36.756 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:36.756 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2019-12-22T22:18:36.756 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:12.2.12-835.ga6ab182.el7 ceph 2.5 M 2019-12-22T22:18:36.756 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:36.756 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2019-12-22T22:18:36.756 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:36.756 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2019-12-22T22:18:36.757 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:36.757 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 2.5 M 2019-12-22T22:18:36.757 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 8.7 M 2019-12-22T22:18:36.757 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading packages: 2019-12-22T22:18:36.855 INFO:teuthology.orchestra.run.smithi043.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:37.044 INFO:teuthology.orchestra.run.smithi091.stdout:Resolving Dependencies 2019-12-22T22:18:37.044 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:18:37.044 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package libcephfs-devel.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:37.044 INFO:teuthology.orchestra.run.smithi091.stdout:--> Processing Dependency: librados-devel = 2:12.2.12-835.ga6ab182.el7 for package: 2:libcephfs-devel-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:37.074 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:18:37.075 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package librados-devel.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:37.207 INFO:teuthology.orchestra.run.smithi043.stdout:Package 2:ceph-mgr-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:18:37.208 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do 2019-12-22T22:18:37.258 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2019-12-22T22:18:37.270 INFO:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-fuse 2019-12-22T22:18:37.282 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2019-12-22T22:18:37.283 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded 2019-12-22T22:18:37.284 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2019-12-22T22:18:37.445 INFO:teuthology.orchestra.run.smithi091.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:37.492 INFO:teuthology.orchestra.run.smithi043.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:37.538 INFO:teuthology.orchestra.run.smithi043.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:37.539 INFO:teuthology.orchestra.run.smithi043.stdout: * base: mirror.umd.edu 2019-12-22T22:18:37.540 INFO:teuthology.orchestra.run.smithi043.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:37.540 INFO:teuthology.orchestra.run.smithi043.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:37.541 INFO:teuthology.orchestra.run.smithi043.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:37.669 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:37.670 INFO:teuthology.orchestra.run.smithi091.stdout:Dependencies Resolved 2019-12-22T22:18:37.671 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:37.671 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:37.671 INFO:teuthology.orchestra.run.smithi091.stdout: Package Arch Version Repository 2019-12-22T22:18:37.672 INFO:teuthology.orchestra.run.smithi091.stdout: Size 2019-12-22T22:18:37.672 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:37.672 INFO:teuthology.orchestra.run.smithi091.stdout:Installing: 2019-12-22T22:18:37.672 INFO:teuthology.orchestra.run.smithi091.stdout: libcephfs-devel x86_64 2:12.2.12-835.ga6ab182.el7 ceph 16 k 2019-12-22T22:18:37.672 INFO:teuthology.orchestra.run.smithi091.stdout:Installing for dependencies: 2019-12-22T22:18:37.672 INFO:teuthology.orchestra.run.smithi091.stdout: librados-devel x86_64 2:12.2.12-835.ga6ab182.el7 ceph 80 k 2019-12-22T22:18:37.673 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:37.673 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction Summary 2019-12-22T22:18:37.673 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:37.673 INFO:teuthology.orchestra.run.smithi091.stdout:Install 1 Package (+1 Dependent package) 2019-12-22T22:18:37.673 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:37.673 INFO:teuthology.orchestra.run.smithi091.stdout:Total download size: 96 k 2019-12-22T22:18:37.674 INFO:teuthology.orchestra.run.smithi091.stdout:Installed size: 385 k 2019-12-22T22:18:37.674 INFO:teuthology.orchestra.run.smithi091.stdout:Downloading packages: 2019-12-22T22:18:38.066 INFO:teuthology.orchestra.run.smithi091.stdout:-------------------------------------------------------------------------------- 2019-12-22T22:18:38.066 INFO:teuthology.orchestra.run.smithi091.stdout:Total 243 kB/s | 96 kB 00:00 2019-12-22T22:18:38.069 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction check 2019-12-22T22:18:38.088 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction test 2019-12-22T22:18:38.090 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction test succeeded 2019-12-22T22:18:38.091 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction 2019-12-22T22:18:38.211 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:ceph-fuse-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:38.565 INFO:teuthology.orchestra.run.smithi043.stdout:Resolving Dependencies 2019-12-22T22:18:38.566 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:18:38.566 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package ceph-fuse.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:38.963 INFO:teuthology.orchestra.run.smithi043.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:38.994 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:ceph-fuse-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:38.994 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:38.994 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2019-12-22T22:18:38.994 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:38.994 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:38.995 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2019-12-22T22:18:39.103 INFO:teuthology.orchestra.run.smithi107:> sudo yum -y install libcephfs2 2019-12-22T22:18:39.153 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:39.154 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies Resolved 2019-12-22T22:18:39.154 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:39.154 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:39.154 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2019-12-22T22:18:39.154 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:39.154 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2019-12-22T22:18:39.155 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse x86_64 2:12.2.12-835.ga6ab182.el7 ceph 2.5 M 2019-12-22T22:18:39.155 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:39.155 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2019-12-22T22:18:39.155 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:39.155 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2019-12-22T22:18:39.155 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:39.155 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 2.5 M 2019-12-22T22:18:39.155 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 8.7 M 2019-12-22T22:18:39.156 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading packages: 2019-12-22T22:18:39.285 INFO:teuthology.orchestra.run.smithi107.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:39.321 INFO:teuthology.orchestra.run.smithi107.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:39.321 INFO:teuthology.orchestra.run.smithi107.stdout: * base: mirror.umd.edu 2019-12-22T22:18:39.322 INFO:teuthology.orchestra.run.smithi107.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:39.322 INFO:teuthology.orchestra.run.smithi107.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:39.322 INFO:teuthology.orchestra.run.smithi107.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:39.606 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:librados-devel-12.2.12-835.ga6ab182.el7.x86_64 1/2 2019-12-22T22:18:39.611 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2019-12-22T22:18:39.638 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2019-12-22T22:18:39.640 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded 2019-12-22T22:18:39.640 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2019-12-22T22:18:39.876 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:libcephfs-devel-12.2.12-835.ga6ab182.el7.x86_64 2/2 2019-12-22T22:18:39.909 INFO:teuthology.orchestra.run.smithi107.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:40.017 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:libcephfs-devel-12.2.12-835.ga6ab182.el7.x86_64 1/2 2019-12-22T22:18:40.235 INFO:teuthology.orchestra.run.smithi107.stdout:Package 2:libcephfs2-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:18:40.235 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do 2019-12-22T22:18:40.295 INFO:teuthology.orchestra.run.smithi107:> sudo yum -y install libcephfs-devel 2019-12-22T22:18:40.490 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:librados-devel-12.2.12-835.ga6ab182.el7.x86_64 2/2 2019-12-22T22:18:40.491 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:40.491 INFO:teuthology.orchestra.run.smithi091.stdout:Installed: 2019-12-22T22:18:40.492 INFO:teuthology.orchestra.run.smithi091.stdout: libcephfs-devel.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:40.492 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:40.492 INFO:teuthology.orchestra.run.smithi091.stdout:Dependency Installed: 2019-12-22T22:18:40.492 INFO:teuthology.orchestra.run.smithi091.stdout: librados-devel.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:40.493 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:40.493 INFO:teuthology.orchestra.run.smithi091.stdout:Complete! 2019-12-22T22:18:40.530 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:ceph-fuse-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:40.545 INFO:teuthology.orchestra.run.smithi107.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:40.590 INFO:teuthology.orchestra.run.smithi107.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:40.591 INFO:teuthology.orchestra.run.smithi107.stdout: * base: mirror.umd.edu 2019-12-22T22:18:40.592 INFO:teuthology.orchestra.run.smithi107.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:40.592 INFO:teuthology.orchestra.run.smithi107.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:40.594 INFO:teuthology.orchestra.run.smithi107.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:40.607 INFO:teuthology.orchestra.run.smithi091:> sudo yum -y install librados2 2019-12-22T22:18:40.827 INFO:teuthology.orchestra.run.smithi091.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:40.867 INFO:teuthology.orchestra.run.smithi091.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:40.867 INFO:teuthology.orchestra.run.smithi091.stdout: * base: mirror.umd.edu 2019-12-22T22:18:40.868 INFO:teuthology.orchestra.run.smithi091.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:40.868 INFO:teuthology.orchestra.run.smithi091.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:40.869 INFO:teuthology.orchestra.run.smithi091.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:41.080 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:ceph-fuse-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:41.080 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:41.080 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2019-12-22T22:18:41.080 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:41.080 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:41.081 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2019-12-22T22:18:41.214 INFO:teuthology.orchestra.run.smithi043:> sudo yum -y install libcephfs2 2019-12-22T22:18:41.246 INFO:teuthology.orchestra.run.smithi107.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:41.480 INFO:teuthology.orchestra.run.smithi043.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:41.511 INFO:teuthology.orchestra.run.smithi091.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:41.526 INFO:teuthology.orchestra.run.smithi043.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:41.527 INFO:teuthology.orchestra.run.smithi043.stdout: * base: mirror.umd.edu 2019-12-22T22:18:41.528 INFO:teuthology.orchestra.run.smithi043.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:41.528 INFO:teuthology.orchestra.run.smithi043.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:41.528 INFO:teuthology.orchestra.run.smithi043.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:41.582 INFO:teuthology.orchestra.run.smithi107.stdout:Resolving Dependencies 2019-12-22T22:18:41.582 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:18:41.582 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package libcephfs-devel.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:41.583 INFO:teuthology.orchestra.run.smithi107.stdout:--> Processing Dependency: librados-devel = 2:12.2.12-835.ga6ab182.el7 for package: 2:libcephfs-devel-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:41.613 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:18:41.613 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package librados-devel.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:41.837 INFO:teuthology.orchestra.run.smithi091.stdout:Package 2:librados2-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:18:41.837 INFO:teuthology.orchestra.run.smithi091.stdout:Nothing to do 2019-12-22T22:18:41.897 INFO:teuthology.orchestra.run.smithi091:> sudo yum -y install librbd1 2019-12-22T22:18:41.984 INFO:teuthology.orchestra.run.smithi107.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:42.078 INFO:teuthology.orchestra.run.smithi091.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:42.112 INFO:teuthology.orchestra.run.smithi091.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:42.117 INFO:teuthology.orchestra.run.smithi091.stdout: * base: mirror.umd.edu 2019-12-22T22:18:42.117 INFO:teuthology.orchestra.run.smithi091.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:42.117 INFO:teuthology.orchestra.run.smithi091.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:42.117 INFO:teuthology.orchestra.run.smithi091.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:42.195 INFO:teuthology.orchestra.run.smithi043.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:42.209 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:42.209 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies Resolved 2019-12-22T22:18:42.210 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:42.211 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:42.211 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository 2019-12-22T22:18:42.211 INFO:teuthology.orchestra.run.smithi107.stdout: Size 2019-12-22T22:18:42.211 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:42.211 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2019-12-22T22:18:42.212 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:12.2.12-835.ga6ab182.el7 ceph 16 k 2019-12-22T22:18:42.212 INFO:teuthology.orchestra.run.smithi107.stdout:Installing for dependencies: 2019-12-22T22:18:42.212 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:12.2.12-835.ga6ab182.el7 ceph 80 k 2019-12-22T22:18:42.212 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:42.212 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2019-12-22T22:18:42.212 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:42.213 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package (+1 Dependent package) 2019-12-22T22:18:42.213 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:42.213 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 96 k 2019-12-22T22:18:42.213 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 385 k 2019-12-22T22:18:42.213 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading packages: 2019-12-22T22:18:42.548 INFO:teuthology.orchestra.run.smithi043.stdout:Package 2:libcephfs2-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:18:42.548 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do 2019-12-22T22:18:42.552 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2019-12-22T22:18:42.552 INFO:teuthology.orchestra.run.smithi107.stdout:Total 281 kB/s | 96 kB 00:00 2019-12-22T22:18:42.555 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2019-12-22T22:18:42.567 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2019-12-22T22:18:42.568 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded 2019-12-22T22:18:42.569 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2019-12-22T22:18:42.613 INFO:teuthology.orchestra.run.smithi043:> sudo yum -y install libcephfs-devel 2019-12-22T22:18:42.703 INFO:teuthology.orchestra.run.smithi091.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:42.867 INFO:teuthology.orchestra.run.smithi043.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:42.912 INFO:teuthology.orchestra.run.smithi043.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:42.913 INFO:teuthology.orchestra.run.smithi043.stdout: * base: mirror.umd.edu 2019-12-22T22:18:42.913 INFO:teuthology.orchestra.run.smithi043.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:42.913 INFO:teuthology.orchestra.run.smithi043.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:42.914 INFO:teuthology.orchestra.run.smithi043.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:43.026 INFO:teuthology.orchestra.run.smithi091.stdout:Package 2:librbd1-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:18:43.026 INFO:teuthology.orchestra.run.smithi091.stdout:Nothing to do 2019-12-22T22:18:43.086 INFO:teuthology.orchestra.run.smithi091:> sudo yum -y install rbd-fuse 2019-12-22T22:18:43.213 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:librados-devel-12.2.12-835.ga6ab182.el7.x86_64 1/2 2019-12-22T22:18:43.333 INFO:teuthology.orchestra.run.smithi091.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:43.378 INFO:teuthology.orchestra.run.smithi091.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:43.379 INFO:teuthology.orchestra.run.smithi091.stdout: * base: mirror.umd.edu 2019-12-22T22:18:43.379 INFO:teuthology.orchestra.run.smithi091.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:43.380 INFO:teuthology.orchestra.run.smithi091.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:43.380 INFO:teuthology.orchestra.run.smithi091.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:43.594 INFO:teuthology.orchestra.run.smithi043.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:43.944 INFO:teuthology.orchestra.run.smithi043.stdout:Resolving Dependencies 2019-12-22T22:18:43.944 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:18:43.945 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package libcephfs-devel.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:43.945 INFO:teuthology.orchestra.run.smithi043.stdout:--> Processing Dependency: librados-devel = 2:12.2.12-835.ga6ab182.el7 for package: 2:libcephfs-devel-12.2.12-835.ga6ab182.el7.x86_64 2019-12-22T22:18:43.977 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:18:43.977 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package librados-devel.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:44.027 INFO:teuthology.orchestra.run.smithi091.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:44.364 INFO:teuthology.orchestra.run.smithi091.stdout:Resolving Dependencies 2019-12-22T22:18:44.365 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:18:44.365 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package rbd-fuse.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:44.379 INFO:teuthology.orchestra.run.smithi043.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:44.419 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:libcephfs-devel-12.2.12-835.ga6ab182.el7.x86_64 2/2 2019-12-22T22:18:44.498 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:libcephfs-devel-12.2.12-835.ga6ab182.el7.x86_64 1/2 2019-12-22T22:18:44.613 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:44.613 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies Resolved 2019-12-22T22:18:44.614 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:44.614 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:44.615 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository 2019-12-22T22:18:44.615 INFO:teuthology.orchestra.run.smithi043.stdout: Size 2019-12-22T22:18:44.616 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:44.616 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2019-12-22T22:18:44.616 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel x86_64 2:12.2.12-835.ga6ab182.el7 ceph 16 k 2019-12-22T22:18:44.616 INFO:teuthology.orchestra.run.smithi043.stdout:Installing for dependencies: 2019-12-22T22:18:44.616 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel x86_64 2:12.2.12-835.ga6ab182.el7 ceph 80 k 2019-12-22T22:18:44.616 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:44.617 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2019-12-22T22:18:44.617 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:44.617 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package (+1 Dependent package) 2019-12-22T22:18:44.617 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:44.617 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 96 k 2019-12-22T22:18:44.617 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 385 k 2019-12-22T22:18:44.618 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading packages: 2019-12-22T22:18:44.733 INFO:teuthology.orchestra.run.smithi091.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:44.936 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:44.937 INFO:teuthology.orchestra.run.smithi091.stdout:Dependencies Resolved 2019-12-22T22:18:44.937 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:44.937 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:44.937 INFO:teuthology.orchestra.run.smithi091.stdout: Package Arch Version Repository Size 2019-12-22T22:18:44.937 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:44.938 INFO:teuthology.orchestra.run.smithi091.stdout:Installing: 2019-12-22T22:18:44.938 INFO:teuthology.orchestra.run.smithi091.stdout: rbd-fuse x86_64 2:12.2.12-835.ga6ab182.el7 ceph 14 k 2019-12-22T22:18:44.938 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:44.938 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction Summary 2019-12-22T22:18:44.939 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:44.939 INFO:teuthology.orchestra.run.smithi091.stdout:Install 1 Package 2019-12-22T22:18:44.939 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:44.939 INFO:teuthology.orchestra.run.smithi091.stdout:Total download size: 14 k 2019-12-22T22:18:44.939 INFO:teuthology.orchestra.run.smithi091.stdout:Installed size: 25 k 2019-12-22T22:18:44.940 INFO:teuthology.orchestra.run.smithi091.stdout:Downloading packages: 2019-12-22T22:18:44.958 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:librados-devel-12.2.12-835.ga6ab182.el7.x86_64 2/2 2019-12-22T22:18:44.958 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:44.958 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2019-12-22T22:18:44.958 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:44.959 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:44.959 INFO:teuthology.orchestra.run.smithi107.stdout:Dependency Installed: 2019-12-22T22:18:44.959 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:44.959 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:44.959 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2019-12-22T22:18:44.976 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2019-12-22T22:18:44.976 INFO:teuthology.orchestra.run.smithi043.stdout:Total 266 kB/s | 96 kB 00:00 2019-12-22T22:18:44.981 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2019-12-22T22:18:45.014 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2019-12-22T22:18:45.014 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded 2019-12-22T22:18:45.014 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2019-12-22T22:18:45.100 INFO:teuthology.orchestra.run.smithi107:> sudo yum -y install librados2 2019-12-22T22:18:45.241 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction check 2019-12-22T22:18:45.253 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction test 2019-12-22T22:18:45.253 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction test succeeded 2019-12-22T22:18:45.253 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction 2019-12-22T22:18:45.394 INFO:teuthology.orchestra.run.smithi107.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:45.439 INFO:teuthology.orchestra.run.smithi107.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:45.440 INFO:teuthology.orchestra.run.smithi107.stdout: * base: mirror.umd.edu 2019-12-22T22:18:45.441 INFO:teuthology.orchestra.run.smithi107.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:45.441 INFO:teuthology.orchestra.run.smithi107.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:45.441 INFO:teuthology.orchestra.run.smithi107.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:45.827 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:rbd-fuse-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:46.079 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:librados-devel-12.2.12-835.ga6ab182.el7.x86_64 1/2 2019-12-22T22:18:46.095 INFO:teuthology.orchestra.run.smithi107.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:46.318 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:rbd-fuse-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:46.319 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:46.319 INFO:teuthology.orchestra.run.smithi091.stdout:Installed: 2019-12-22T22:18:46.319 INFO:teuthology.orchestra.run.smithi091.stdout: rbd-fuse.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:46.319 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:46.319 INFO:teuthology.orchestra.run.smithi091.stdout:Complete! 2019-12-22T22:18:46.330 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:libcephfs-devel-12.2.12-835.ga6ab182.el7.x86_64 2/2 2019-12-22T22:18:46.431 INFO:teuthology.orchestra.run.smithi107.stdout:Package 2:librados2-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:18:46.431 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do 2019-12-22T22:18:46.434 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:libcephfs-devel-12.2.12-835.ga6ab182.el7.x86_64 1/2 2019-12-22T22:18:46.441 INFO:teuthology.orchestra.run.smithi091:> sudo yum -y install rbd-mirror 2019-12-22T22:18:46.491 INFO:teuthology.orchestra.run.smithi107:> sudo yum -y install librbd1 2019-12-22T22:18:46.648 INFO:teuthology.orchestra.run.smithi091.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:46.652 INFO:teuthology.orchestra.run.smithi107.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:46.687 INFO:teuthology.orchestra.run.smithi107.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:46.688 INFO:teuthology.orchestra.run.smithi091.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:46.688 INFO:teuthology.orchestra.run.smithi107.stdout: * base: mirror.umd.edu 2019-12-22T22:18:46.688 INFO:teuthology.orchestra.run.smithi107.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:46.689 INFO:teuthology.orchestra.run.smithi107.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:46.689 INFO:teuthology.orchestra.run.smithi107.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:46.689 INFO:teuthology.orchestra.run.smithi091.stdout: * base: mirror.umd.edu 2019-12-22T22:18:46.690 INFO:teuthology.orchestra.run.smithi091.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:46.690 INFO:teuthology.orchestra.run.smithi091.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:46.690 INFO:teuthology.orchestra.run.smithi091.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:46.910 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:librados-devel-12.2.12-835.ga6ab182.el7.x86_64 2/2 2019-12-22T22:18:46.911 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:46.911 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2019-12-22T22:18:46.911 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:46.911 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:46.912 INFO:teuthology.orchestra.run.smithi043.stdout:Dependency Installed: 2019-12-22T22:18:46.912 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:46.912 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:46.912 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2019-12-22T22:18:47.049 INFO:teuthology.orchestra.run.smithi043:> sudo yum -y install librados2 2019-12-22T22:18:47.281 INFO:teuthology.orchestra.run.smithi107.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:47.321 INFO:teuthology.orchestra.run.smithi043.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:47.333 INFO:teuthology.orchestra.run.smithi091.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:47.369 INFO:teuthology.orchestra.run.smithi043.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:47.370 INFO:teuthology.orchestra.run.smithi043.stdout: * base: mirror.umd.edu 2019-12-22T22:18:47.370 INFO:teuthology.orchestra.run.smithi043.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:47.370 INFO:teuthology.orchestra.run.smithi043.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:47.371 INFO:teuthology.orchestra.run.smithi043.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:47.602 INFO:teuthology.orchestra.run.smithi107.stdout:Package 2:librbd1-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:18:47.602 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do 2019-12-22T22:18:47.659 INFO:teuthology.orchestra.run.smithi091.stdout:Resolving Dependencies 2019-12-22T22:18:47.660 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:18:47.660 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package rbd-mirror.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:47.662 INFO:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-fuse 2019-12-22T22:18:47.889 INFO:teuthology.orchestra.run.smithi107.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:47.931 INFO:teuthology.orchestra.run.smithi107.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:47.932 INFO:teuthology.orchestra.run.smithi107.stdout: * base: mirror.umd.edu 2019-12-22T22:18:47.933 INFO:teuthology.orchestra.run.smithi107.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:47.933 INFO:teuthology.orchestra.run.smithi107.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:47.933 INFO:teuthology.orchestra.run.smithi107.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:48.035 INFO:teuthology.orchestra.run.smithi091.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:48.056 INFO:teuthology.orchestra.run.smithi043.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:48.257 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:48.257 INFO:teuthology.orchestra.run.smithi091.stdout:Dependencies Resolved 2019-12-22T22:18:48.258 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:48.258 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:48.258 INFO:teuthology.orchestra.run.smithi091.stdout: Package Arch Version Repository 2019-12-22T22:18:48.259 INFO:teuthology.orchestra.run.smithi091.stdout: Size 2019-12-22T22:18:48.259 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:48.259 INFO:teuthology.orchestra.run.smithi091.stdout:Installing: 2019-12-22T22:18:48.259 INFO:teuthology.orchestra.run.smithi091.stdout: rbd-mirror x86_64 2:12.2.12-835.ga6ab182.el7 ceph 1.5 M 2019-12-22T22:18:48.259 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:48.259 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction Summary 2019-12-22T22:18:48.259 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:48.260 INFO:teuthology.orchestra.run.smithi091.stdout:Install 1 Package 2019-12-22T22:18:48.260 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:48.260 INFO:teuthology.orchestra.run.smithi091.stdout:Total download size: 1.5 M 2019-12-22T22:18:48.260 INFO:teuthology.orchestra.run.smithi091.stdout:Installed size: 5.9 M 2019-12-22T22:18:48.261 INFO:teuthology.orchestra.run.smithi091.stdout:Downloading packages: 2019-12-22T22:18:48.402 INFO:teuthology.orchestra.run.smithi043.stdout:Package 2:librados2-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:18:48.402 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do 2019-12-22T22:18:48.463 INFO:teuthology.orchestra.run.smithi043:> sudo yum -y install librbd1 2019-12-22T22:18:48.580 INFO:teuthology.orchestra.run.smithi107.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:48.638 INFO:teuthology.orchestra.run.smithi043.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:48.664 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction check 2019-12-22T22:18:48.679 INFO:teuthology.orchestra.run.smithi043.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:48.680 INFO:teuthology.orchestra.run.smithi043.stdout: * base: mirror.umd.edu 2019-12-22T22:18:48.680 INFO:teuthology.orchestra.run.smithi043.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:48.680 INFO:teuthology.orchestra.run.smithi043.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:48.680 INFO:teuthology.orchestra.run.smithi043.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:48.685 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction test 2019-12-22T22:18:48.686 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction test succeeded 2019-12-22T22:18:48.687 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction 2019-12-22T22:18:48.910 INFO:teuthology.orchestra.run.smithi107.stdout:Resolving Dependencies 2019-12-22T22:18:48.911 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:18:48.911 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package rbd-fuse.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:49.284 INFO:teuthology.orchestra.run.smithi107.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:49.304 INFO:teuthology.orchestra.run.smithi043.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:49.467 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:49.468 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies Resolved 2019-12-22T22:18:49.468 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:49.468 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:49.468 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2019-12-22T22:18:49.469 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:49.469 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2019-12-22T22:18:49.469 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:12.2.12-835.ga6ab182.el7 ceph 14 k 2019-12-22T22:18:49.469 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:49.470 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2019-12-22T22:18:49.470 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:49.470 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2019-12-22T22:18:49.470 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:49.470 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 14 k 2019-12-22T22:18:49.471 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 25 k 2019-12-22T22:18:49.471 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading packages: 2019-12-22T22:18:49.648 INFO:teuthology.orchestra.run.smithi043.stdout:Package 2:librbd1-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:18:49.648 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do 2019-12-22T22:18:49.710 INFO:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-fuse 2019-12-22T22:18:49.784 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2019-12-22T22:18:49.793 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2019-12-22T22:18:49.794 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded 2019-12-22T22:18:49.795 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2019-12-22T22:18:49.920 INFO:teuthology.orchestra.run.smithi043.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:49.966 INFO:teuthology.orchestra.run.smithi043.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:49.966 INFO:teuthology.orchestra.run.smithi043.stdout: * base: mirror.umd.edu 2019-12-22T22:18:49.967 INFO:teuthology.orchestra.run.smithi043.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:49.968 INFO:teuthology.orchestra.run.smithi043.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:49.968 INFO:teuthology.orchestra.run.smithi043.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:49.995 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:rbd-mirror-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:50.425 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:rbd-mirror-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:50.425 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:50.425 INFO:teuthology.orchestra.run.smithi091.stdout:Installed: 2019-12-22T22:18:50.425 INFO:teuthology.orchestra.run.smithi091.stdout: rbd-mirror.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:50.426 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:50.426 INFO:teuthology.orchestra.run.smithi091.stdout:Complete! 2019-12-22T22:18:50.443 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:rbd-fuse-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:50.538 INFO:teuthology.orchestra.run.smithi091:> sudo yum -y install rbd-nbd 2019-12-22T22:18:50.634 INFO:teuthology.orchestra.run.smithi043.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:50.781 INFO:teuthology.orchestra.run.smithi091.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:50.823 INFO:teuthology.orchestra.run.smithi091.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:50.824 INFO:teuthology.orchestra.run.smithi091.stdout: * base: mirror.umd.edu 2019-12-22T22:18:50.824 INFO:teuthology.orchestra.run.smithi091.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:50.824 INFO:teuthology.orchestra.run.smithi091.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:50.825 INFO:teuthology.orchestra.run.smithi091.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:50.946 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:rbd-fuse-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:50.947 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:50.947 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2019-12-22T22:18:50.947 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:50.947 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:50.947 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2019-12-22T22:18:50.985 INFO:teuthology.orchestra.run.smithi043.stdout:Resolving Dependencies 2019-12-22T22:18:50.985 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:18:50.985 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package rbd-fuse.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:51.068 INFO:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-mirror 2019-12-22T22:18:51.254 INFO:teuthology.orchestra.run.smithi107.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:51.297 INFO:teuthology.orchestra.run.smithi107.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:51.297 INFO:teuthology.orchestra.run.smithi107.stdout: * base: mirror.umd.edu 2019-12-22T22:18:51.297 INFO:teuthology.orchestra.run.smithi107.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:51.297 INFO:teuthology.orchestra.run.smithi107.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:51.298 INFO:teuthology.orchestra.run.smithi107.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:51.378 INFO:teuthology.orchestra.run.smithi043.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:51.473 INFO:teuthology.orchestra.run.smithi091.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:51.568 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:51.569 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies Resolved 2019-12-22T22:18:51.569 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:51.570 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:51.570 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2019-12-22T22:18:51.570 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:51.571 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2019-12-22T22:18:51.571 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse x86_64 2:12.2.12-835.ga6ab182.el7 ceph 14 k 2019-12-22T22:18:51.571 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:51.571 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2019-12-22T22:18:51.571 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:51.571 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2019-12-22T22:18:51.571 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:51.572 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 14 k 2019-12-22T22:18:51.572 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 25 k 2019-12-22T22:18:51.572 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading packages: 2019-12-22T22:18:51.809 INFO:teuthology.orchestra.run.smithi091.stdout:Resolving Dependencies 2019-12-22T22:18:51.809 INFO:teuthology.orchestra.run.smithi091.stdout:--> Running transaction check 2019-12-22T22:18:51.809 INFO:teuthology.orchestra.run.smithi091.stdout:---> Package rbd-nbd.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:51.890 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2019-12-22T22:18:51.903 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2019-12-22T22:18:51.905 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded 2019-12-22T22:18:51.905 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2019-12-22T22:18:51.926 INFO:teuthology.orchestra.run.smithi107.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:52.182 INFO:teuthology.orchestra.run.smithi091.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:52.252 INFO:teuthology.orchestra.run.smithi107.stdout:Resolving Dependencies 2019-12-22T22:18:52.252 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:18:52.252 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package rbd-mirror.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:52.363 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:52.363 INFO:teuthology.orchestra.run.smithi091.stdout:Dependencies Resolved 2019-12-22T22:18:52.364 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:52.365 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:52.365 INFO:teuthology.orchestra.run.smithi091.stdout: Package Arch Version Repository Size 2019-12-22T22:18:52.365 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:52.365 INFO:teuthology.orchestra.run.smithi091.stdout:Installing: 2019-12-22T22:18:52.366 INFO:teuthology.orchestra.run.smithi091.stdout: rbd-nbd x86_64 2:12.2.12-835.ga6ab182.el7 ceph 82 k 2019-12-22T22:18:52.366 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:52.366 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction Summary 2019-12-22T22:18:52.366 INFO:teuthology.orchestra.run.smithi091.stdout:================================================================================ 2019-12-22T22:18:52.366 INFO:teuthology.orchestra.run.smithi091.stdout:Install 1 Package 2019-12-22T22:18:52.367 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:52.367 INFO:teuthology.orchestra.run.smithi091.stdout:Total download size: 82 k 2019-12-22T22:18:52.367 INFO:teuthology.orchestra.run.smithi091.stdout:Installed size: 228 k 2019-12-22T22:18:52.367 INFO:teuthology.orchestra.run.smithi091.stdout:Downloading packages: 2019-12-22T22:18:52.547 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:rbd-fuse-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:52.628 INFO:teuthology.orchestra.run.smithi107.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:52.698 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction check 2019-12-22T22:18:52.717 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction test 2019-12-22T22:18:52.718 INFO:teuthology.orchestra.run.smithi091.stdout:Transaction test succeeded 2019-12-22T22:18:52.719 INFO:teuthology.orchestra.run.smithi091.stdout:Running transaction 2019-12-22T22:18:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:52.845 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies Resolved 2019-12-22T22:18:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:52.845 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository 2019-12-22T22:18:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: Size 2019-12-22T22:18:52.846 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:52.846 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2019-12-22T22:18:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:12.2.12-835.ga6ab182.el7 ceph 1.5 M 2019-12-22T22:18:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:52.846 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2019-12-22T22:18:52.847 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:52.847 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2019-12-22T22:18:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:52.847 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 1.5 M 2019-12-22T22:18:52.847 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 5.9 M 2019-12-22T22:18:52.847 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading packages: 2019-12-22T22:18:52.947 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:rbd-fuse-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:52.947 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:52.947 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2019-12-22T22:18:52.947 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:52.948 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:52.948 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2019-12-22T22:18:53.069 INFO:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-mirror 2019-12-22T22:18:53.252 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2019-12-22T22:18:53.273 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2019-12-22T22:18:53.274 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded 2019-12-22T22:18:53.275 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2019-12-22T22:18:53.310 INFO:teuthology.orchestra.run.smithi091.stdout: Installing : 2:rbd-nbd-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:53.319 INFO:teuthology.orchestra.run.smithi043.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:53.362 INFO:teuthology.orchestra.run.smithi043.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:53.363 INFO:teuthology.orchestra.run.smithi043.stdout: * base: mirror.umd.edu 2019-12-22T22:18:53.364 INFO:teuthology.orchestra.run.smithi043.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:53.364 INFO:teuthology.orchestra.run.smithi043.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:53.364 INFO:teuthology.orchestra.run.smithi043.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:53.844 INFO:teuthology.orchestra.run.smithi091.stdout: Verifying : 2:rbd-nbd-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:53.844 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:53.844 INFO:teuthology.orchestra.run.smithi091.stdout:Installed: 2019-12-22T22:18:53.844 INFO:teuthology.orchestra.run.smithi091.stdout: rbd-nbd.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:53.844 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:18:53.844 INFO:teuthology.orchestra.run.smithi091.stdout:Complete! 2019-12-22T22:18:53.950 INFO:teuthology.orchestra.run.smithi091:> sudo yum -y install librados2 2019-12-22T22:18:54.052 INFO:teuthology.orchestra.run.smithi043.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:54.180 INFO:teuthology.orchestra.run.smithi091.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:54.210 INFO:teuthology.orchestra.run.smithi091.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:54.211 INFO:teuthology.orchestra.run.smithi091.stdout: * base: mirror.umd.edu 2019-12-22T22:18:54.212 INFO:teuthology.orchestra.run.smithi091.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:54.212 INFO:teuthology.orchestra.run.smithi091.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:54.212 INFO:teuthology.orchestra.run.smithi091.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:54.401 INFO:teuthology.orchestra.run.smithi043.stdout:Resolving Dependencies 2019-12-22T22:18:54.402 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:18:54.402 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package rbd-mirror.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:54.638 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:rbd-mirror-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:54.803 INFO:teuthology.orchestra.run.smithi043.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:54.855 INFO:teuthology.orchestra.run.smithi091.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:55.034 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:55.034 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies Resolved 2019-12-22T22:18:55.035 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:55.035 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:55.035 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository 2019-12-22T22:18:55.035 INFO:teuthology.orchestra.run.smithi043.stdout: Size 2019-12-22T22:18:55.036 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:55.036 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2019-12-22T22:18:55.036 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror x86_64 2:12.2.12-835.ga6ab182.el7 ceph 1.5 M 2019-12-22T22:18:55.036 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:55.036 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2019-12-22T22:18:55.036 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:55.036 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2019-12-22T22:18:55.037 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:55.037 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 1.5 M 2019-12-22T22:18:55.037 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 5.9 M 2019-12-22T22:18:55.037 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading packages: 2019-12-22T22:18:55.157 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:rbd-mirror-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:55.157 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:55.158 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2019-12-22T22:18:55.158 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:55.158 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:55.158 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2019-12-22T22:18:55.178 INFO:teuthology.orchestra.run.smithi091.stdout:Package 2:librados2-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:18:55.179 INFO:teuthology.orchestra.run.smithi091.stdout:Nothing to do 2019-12-22T22:18:55.238 INFO:teuthology.orchestra.run.smithi091:> sudo yum -y install python-rados 2019-12-22T22:18:55.277 INFO:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-nbd 2019-12-22T22:18:55.454 INFO:teuthology.orchestra.run.smithi091.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:55.481 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2019-12-22T22:18:55.500 INFO:teuthology.orchestra.run.smithi091.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:55.500 INFO:teuthology.orchestra.run.smithi091.stdout: * base: mirror.umd.edu 2019-12-22T22:18:55.501 INFO:teuthology.orchestra.run.smithi091.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:55.501 INFO:teuthology.orchestra.run.smithi091.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:55.501 INFO:teuthology.orchestra.run.smithi091.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:55.502 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2019-12-22T22:18:55.502 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded 2019-12-22T22:18:55.502 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2019-12-22T22:18:55.537 INFO:teuthology.orchestra.run.smithi107.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:55.584 INFO:teuthology.orchestra.run.smithi107.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:55.585 INFO:teuthology.orchestra.run.smithi107.stdout: * base: mirror.umd.edu 2019-12-22T22:18:55.585 INFO:teuthology.orchestra.run.smithi107.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:55.585 INFO:teuthology.orchestra.run.smithi107.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:55.585 INFO:teuthology.orchestra.run.smithi107.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:56.140 INFO:teuthology.orchestra.run.smithi091.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:56.226 INFO:teuthology.orchestra.run.smithi107.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:56.356 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:rbd-mirror-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:56.468 INFO:teuthology.orchestra.run.smithi091.stdout:Package 2:python-rados-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:18:56.468 INFO:teuthology.orchestra.run.smithi091.stdout:Nothing to do 2019-12-22T22:18:56.529 INFO:teuthology.orchestra.run.smithi091:> sudo yum -y install python-rgw 2019-12-22T22:18:56.565 INFO:teuthology.orchestra.run.smithi107.stdout:Resolving Dependencies 2019-12-22T22:18:56.565 INFO:teuthology.orchestra.run.smithi107.stdout:--> Running transaction check 2019-12-22T22:18:56.565 INFO:teuthology.orchestra.run.smithi107.stdout:---> Package rbd-nbd.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:56.773 INFO:teuthology.orchestra.run.smithi091.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:56.815 INFO:teuthology.orchestra.run.smithi091.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:56.816 INFO:teuthology.orchestra.run.smithi091.stdout: * base: mirror.umd.edu 2019-12-22T22:18:56.817 INFO:teuthology.orchestra.run.smithi091.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:56.817 INFO:teuthology.orchestra.run.smithi091.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:56.817 INFO:teuthology.orchestra.run.smithi091.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:56.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:rbd-mirror-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:56.861 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:56.861 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2019-12-22T22:18:56.862 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:56.862 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:56.862 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2019-12-22T22:18:56.934 INFO:teuthology.orchestra.run.smithi107.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:56.989 INFO:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-nbd 2019-12-22T22:18:57.114 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:57.114 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies Resolved 2019-12-22T22:18:57.115 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:57.115 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:57.115 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2019-12-22T22:18:57.115 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:57.115 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2019-12-22T22:18:57.115 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:12.2.12-835.ga6ab182.el7 ceph 82 k 2019-12-22T22:18:57.116 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:57.116 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2019-12-22T22:18:57.116 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2019-12-22T22:18:57.116 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2019-12-22T22:18:57.116 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:57.116 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 82 k 2019-12-22T22:18:57.117 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 228 k 2019-12-22T22:18:57.117 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading packages: 2019-12-22T22:18:57.183 INFO:teuthology.orchestra.run.smithi043.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:57.223 INFO:teuthology.orchestra.run.smithi043.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:57.223 INFO:teuthology.orchestra.run.smithi043.stdout: * base: mirror.umd.edu 2019-12-22T22:18:57.224 INFO:teuthology.orchestra.run.smithi043.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:57.224 INFO:teuthology.orchestra.run.smithi043.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:57.225 INFO:teuthology.orchestra.run.smithi043.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:57.442 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2019-12-22T22:18:57.461 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2019-12-22T22:18:57.462 INFO:teuthology.orchestra.run.smithi091.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:57.463 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded 2019-12-22T22:18:57.463 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2019-12-22T22:18:57.801 INFO:teuthology.orchestra.run.smithi091.stdout:Package 2:python-rgw-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:18:57.801 INFO:teuthology.orchestra.run.smithi091.stdout:Nothing to do 2019-12-22T22:18:57.862 INFO:teuthology.orchestra.run.smithi091:> sudo yum -y install python-rbd 2019-12-22T22:18:57.878 INFO:teuthology.orchestra.run.smithi043.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:58.094 INFO:teuthology.orchestra.run.smithi091.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:58.120 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : 2:rbd-nbd-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:58.136 INFO:teuthology.orchestra.run.smithi091.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:58.137 INFO:teuthology.orchestra.run.smithi091.stdout: * base: mirror.umd.edu 2019-12-22T22:18:58.138 INFO:teuthology.orchestra.run.smithi091.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:58.138 INFO:teuthology.orchestra.run.smithi091.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:58.139 INFO:teuthology.orchestra.run.smithi091.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:58.224 INFO:teuthology.orchestra.run.smithi043.stdout:Resolving Dependencies 2019-12-22T22:18:58.224 INFO:teuthology.orchestra.run.smithi043.stdout:--> Running transaction check 2019-12-22T22:18:58.225 INFO:teuthology.orchestra.run.smithi043.stdout:---> Package rbd-nbd.x86_64 2:12.2.12-835.ga6ab182.el7 will be installed 2019-12-22T22:18:58.614 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : 2:rbd-nbd-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:58.614 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:58.614 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2019-12-22T22:18:58.615 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:18:58.615 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:18:58.615 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2019-12-22T22:18:58.619 INFO:teuthology.orchestra.run.smithi043.stdout:--> Finished Dependency Resolution 2019-12-22T22:18:58.731 INFO:teuthology.orchestra.run.smithi107:> sudo yum -y install librados2 2019-12-22T22:18:58.788 INFO:teuthology.orchestra.run.smithi091.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:58.811 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:58.811 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies Resolved 2019-12-22T22:18:58.812 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:58.813 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:58.813 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2019-12-22T22:18:58.813 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:58.813 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2019-12-22T22:18:58.813 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd x86_64 2:12.2.12-835.ga6ab182.el7 ceph 82 k 2019-12-22T22:18:58.814 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:58.814 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2019-12-22T22:18:58.814 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2019-12-22T22:18:58.814 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2019-12-22T22:18:58.814 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:18:58.814 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 82 k 2019-12-22T22:18:58.814 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 228 k 2019-12-22T22:18:58.815 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading packages: 2019-12-22T22:18:58.977 INFO:teuthology.orchestra.run.smithi107.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:59.010 INFO:teuthology.orchestra.run.smithi107.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:59.011 INFO:teuthology.orchestra.run.smithi107.stdout: * base: mirror.umd.edu 2019-12-22T22:18:59.012 INFO:teuthology.orchestra.run.smithi107.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:59.012 INFO:teuthology.orchestra.run.smithi107.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:59.012 INFO:teuthology.orchestra.run.smithi107.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:59.120 INFO:teuthology.orchestra.run.smithi091.stdout:Package 2:python-rbd-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:18:59.120 INFO:teuthology.orchestra.run.smithi091.stdout:Nothing to do 2019-12-22T22:18:59.145 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2019-12-22T22:18:59.166 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2019-12-22T22:18:59.168 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded 2019-12-22T22:18:59.168 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2019-12-22T22:18:59.181 INFO:teuthology.orchestra.run.smithi091:> sudo yum -y install python-cephfs 2019-12-22T22:18:59.342 INFO:teuthology.orchestra.run.smithi091.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:18:59.377 INFO:teuthology.orchestra.run.smithi091.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:18:59.377 INFO:teuthology.orchestra.run.smithi091.stdout: * base: mirror.umd.edu 2019-12-22T22:18:59.378 INFO:teuthology.orchestra.run.smithi091.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:18:59.378 INFO:teuthology.orchestra.run.smithi091.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:18:59.378 INFO:teuthology.orchestra.run.smithi091.stdout: * updates: repos-va.psychz.net 2019-12-22T22:18:59.661 INFO:teuthology.orchestra.run.smithi107.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:59.779 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : 2:rbd-nbd-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:18:59.969 INFO:teuthology.orchestra.run.smithi091.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:18:59.988 INFO:teuthology.orchestra.run.smithi107.stdout:Package 2:librados2-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:18:59.989 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do 2019-12-22T22:19:00.047 INFO:teuthology.orchestra.run.smithi107:> sudo yum -y install python-rados 2019-12-22T22:19:00.255 INFO:teuthology.orchestra.run.smithi107.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:19:00.283 INFO:teuthology.orchestra.run.smithi091.stdout:Package 2:python-cephfs-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:19:00.284 INFO:teuthology.orchestra.run.smithi091.stdout:Nothing to do 2019-12-22T22:19:00.292 INFO:teuthology.orchestra.run.smithi107.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:19:00.293 INFO:teuthology.orchestra.run.smithi107.stdout: * base: mirror.umd.edu 2019-12-22T22:19:00.294 INFO:teuthology.orchestra.run.smithi107.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:19:00.294 INFO:teuthology.orchestra.run.smithi107.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:19:00.294 INFO:teuthology.orchestra.run.smithi107.stdout: * updates: repos-va.psychz.net 2019-12-22T22:19:00.344 DEBUG:teuthology.parallel:result is None 2019-12-22T22:19:00.882 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : 2:rbd-nbd-12.2.12-835.ga6ab182.el7.x86_64 1/1 2019-12-22T22:19:00.882 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:19:00.883 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2019-12-22T22:19:00.883 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd.x86_64 2:12.2.12-835.ga6ab182.el7 2019-12-22T22:19:00.883 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:19:00.883 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2019-12-22T22:19:00.884 INFO:teuthology.orchestra.run.smithi107.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:19:01.002 INFO:teuthology.orchestra.run.smithi043:> sudo yum -y install librados2 2019-12-22T22:19:01.198 INFO:teuthology.orchestra.run.smithi043.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:19:01.209 INFO:teuthology.orchestra.run.smithi107.stdout:Package 2:python-rados-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:19:01.209 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do 2019-12-22T22:19:01.228 INFO:teuthology.orchestra.run.smithi043.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:19:01.230 INFO:teuthology.orchestra.run.smithi043.stdout: * base: mirror.umd.edu 2019-12-22T22:19:01.230 INFO:teuthology.orchestra.run.smithi043.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:19:01.230 INFO:teuthology.orchestra.run.smithi043.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:19:01.231 INFO:teuthology.orchestra.run.smithi043.stdout: * updates: repos-va.psychz.net 2019-12-22T22:19:01.271 INFO:teuthology.orchestra.run.smithi107:> sudo yum -y install python-rgw 2019-12-22T22:19:01.529 INFO:teuthology.orchestra.run.smithi107.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:19:01.574 INFO:teuthology.orchestra.run.smithi107.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:19:01.575 INFO:teuthology.orchestra.run.smithi107.stdout: * base: mirror.umd.edu 2019-12-22T22:19:01.576 INFO:teuthology.orchestra.run.smithi107.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:19:01.576 INFO:teuthology.orchestra.run.smithi107.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:19:01.577 INFO:teuthology.orchestra.run.smithi107.stdout: * updates: repos-va.psychz.net 2019-12-22T22:19:01.876 INFO:teuthology.orchestra.run.smithi043.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:19:02.219 INFO:teuthology.orchestra.run.smithi043.stdout:Package 2:librados2-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:19:02.219 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do 2019-12-22T22:19:02.232 INFO:teuthology.orchestra.run.smithi107.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:19:02.281 INFO:teuthology.orchestra.run.smithi043:> sudo yum -y install python-rados 2019-12-22T22:19:02.488 INFO:teuthology.orchestra.run.smithi043.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:19:02.527 INFO:teuthology.orchestra.run.smithi043.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:19:02.528 INFO:teuthology.orchestra.run.smithi043.stdout: * base: mirror.umd.edu 2019-12-22T22:19:02.528 INFO:teuthology.orchestra.run.smithi043.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:19:02.529 INFO:teuthology.orchestra.run.smithi043.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:19:02.529 INFO:teuthology.orchestra.run.smithi043.stdout: * updates: repos-va.psychz.net 2019-12-22T22:19:02.568 INFO:teuthology.orchestra.run.smithi107.stdout:Package 2:python-rgw-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:19:02.568 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do 2019-12-22T22:19:02.629 INFO:teuthology.orchestra.run.smithi107:> sudo yum -y install python-rbd 2019-12-22T22:19:02.869 INFO:teuthology.orchestra.run.smithi107.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:19:02.913 INFO:teuthology.orchestra.run.smithi107.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:19:02.914 INFO:teuthology.orchestra.run.smithi107.stdout: * base: mirror.umd.edu 2019-12-22T22:19:02.915 INFO:teuthology.orchestra.run.smithi107.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:19:02.915 INFO:teuthology.orchestra.run.smithi107.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:19:02.915 INFO:teuthology.orchestra.run.smithi107.stdout: * updates: repos-va.psychz.net 2019-12-22T22:19:03.205 INFO:teuthology.orchestra.run.smithi043.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:19:03.548 INFO:teuthology.orchestra.run.smithi043.stdout:Package 2:python-rados-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:19:03.548 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do 2019-12-22T22:19:03.565 INFO:teuthology.orchestra.run.smithi107.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:19:03.612 INFO:teuthology.orchestra.run.smithi043:> sudo yum -y install python-rgw 2019-12-22T22:19:03.862 INFO:teuthology.orchestra.run.smithi043.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:19:03.902 INFO:teuthology.orchestra.run.smithi107.stdout:Package 2:python-rbd-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:19:03.905 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do 2019-12-22T22:19:03.909 INFO:teuthology.orchestra.run.smithi043.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:19:03.912 INFO:teuthology.orchestra.run.smithi043.stdout: * base: mirror.umd.edu 2019-12-22T22:19:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:19:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:19:03.913 INFO:teuthology.orchestra.run.smithi043.stdout: * updates: repos-va.psychz.net 2019-12-22T22:19:03.961 INFO:teuthology.orchestra.run.smithi107:> sudo yum -y install python-cephfs 2019-12-22T22:19:04.184 INFO:teuthology.orchestra.run.smithi107.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:19:04.222 INFO:teuthology.orchestra.run.smithi107.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:19:04.223 INFO:teuthology.orchestra.run.smithi107.stdout: * base: mirror.umd.edu 2019-12-22T22:19:04.224 INFO:teuthology.orchestra.run.smithi107.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:19:04.225 INFO:teuthology.orchestra.run.smithi107.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:19:04.225 INFO:teuthology.orchestra.run.smithi107.stdout: * updates: repos-va.psychz.net 2019-12-22T22:19:04.572 INFO:teuthology.orchestra.run.smithi043.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:19:04.871 INFO:teuthology.orchestra.run.smithi107.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:19:04.926 INFO:teuthology.orchestra.run.smithi043.stdout:Package 2:python-rgw-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:19:04.926 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do 2019-12-22T22:19:04.988 INFO:teuthology.orchestra.run.smithi043:> sudo yum -y install python-rbd 2019-12-22T22:19:05.198 INFO:teuthology.orchestra.run.smithi107.stdout:Package 2:python-cephfs-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:19:05.198 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do 2019-12-22T22:19:05.207 INFO:teuthology.orchestra.run.smithi043.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:19:05.257 INFO:teuthology.orchestra.run.smithi043.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:19:05.257 INFO:teuthology.orchestra.run.smithi043.stdout: * base: mirror.umd.edu 2019-12-22T22:19:05.257 INFO:teuthology.orchestra.run.smithi043.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:19:05.257 INFO:teuthology.orchestra.run.smithi043.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:19:05.257 INFO:teuthology.orchestra.run.smithi043.stdout: * updates: repos-va.psychz.net 2019-12-22T22:19:05.258 DEBUG:teuthology.parallel:result is None 2019-12-22T22:19:05.945 INFO:teuthology.orchestra.run.smithi043.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:19:06.296 INFO:teuthology.orchestra.run.smithi043.stdout:Package 2:python-rbd-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:19:06.297 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do 2019-12-22T22:19:06.358 INFO:teuthology.orchestra.run.smithi043:> sudo yum -y install python-cephfs 2019-12-22T22:19:06.550 INFO:teuthology.orchestra.run.smithi043.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-22T22:19:06.591 INFO:teuthology.orchestra.run.smithi043.stdout:Loading mirror speeds from cached hostfile 2019-12-22T22:19:06.592 INFO:teuthology.orchestra.run.smithi043.stdout: * base: mirror.umd.edu 2019-12-22T22:19:06.592 INFO:teuthology.orchestra.run.smithi043.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-22T22:19:06.592 INFO:teuthology.orchestra.run.smithi043.stdout: * extras: mirror.fileplanet.com 2019-12-22T22:19:06.593 INFO:teuthology.orchestra.run.smithi043.stdout: * updates: repos-va.psychz.net 2019-12-22T22:19:07.222 INFO:teuthology.orchestra.run.smithi043.stdout:7 packages excluded due to repository priority protections 2019-12-22T22:19:07.563 INFO:teuthology.orchestra.run.smithi043.stdout:Package 2:python-cephfs-12.2.12-835.ga6ab182.el7.x86_64 already installed and latest version 2019-12-22T22:19:07.563 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do 2019-12-22T22:19:07.626 DEBUG:teuthology.parallel:result is None 2019-12-22T22:19:07.626 INFO:teuthology.orchestra.run.smithi091:> sudo lsb_release -is 2019-12-22T22:19:07.776 INFO:teuthology.orchestra.run.smithi091.stdout:CentOS 2019-12-22T22:19:07.778 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-22T22:19:07.779 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2019-12-22T22:19:07.779 INFO:teuthology.packaging:ref: None 2019-12-22T22:19:07.779 INFO:teuthology.packaging:tag: None 2019-12-22T22:19:07.779 INFO:teuthology.packaging:branch: luminous 2019-12-22T22:19:07.779 INFO:teuthology.packaging:sha1: 99b509ffd4dd354919b08b38ba83b998fbfdc107 2019-12-22T22:19:07.779 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F7%2Fx86_64&ref=luminous 2019-12-22T22:19:08.197 INFO:teuthology.orchestra.run.smithi091:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2019-12-22T22:19:08.335 INFO:teuthology.orchestra.run.smithi091.stdout:12.2.12-835.ga6ab182.el7 2019-12-22T22:19:08.335 INFO:teuthology.packaging:The installed version of ceph is 12.2.12-835.ga6ab182.el7 2019-12-22T22:19:08.335 INFO:teuthology.task.install:The correct ceph version 12.2.12-835.ga6ab182 is installed. 2019-12-22T22:19:08.336 INFO:teuthology.orchestra.run.smithi043:> sudo lsb_release -is 2019-12-22T22:19:08.493 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS 2019-12-22T22:19:08.495 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-22T22:19:08.496 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2019-12-22T22:19:08.496 INFO:teuthology.packaging:ref: None 2019-12-22T22:19:08.496 INFO:teuthology.packaging:tag: None 2019-12-22T22:19:08.496 INFO:teuthology.packaging:branch: luminous 2019-12-22T22:19:08.496 INFO:teuthology.packaging:sha1: 99b509ffd4dd354919b08b38ba83b998fbfdc107 2019-12-22T22:19:08.496 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F7%2Fx86_64&ref=luminous 2019-12-22T22:19:08.893 INFO:teuthology.orchestra.run.smithi043:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2019-12-22T22:19:09.027 INFO:teuthology.orchestra.run.smithi043.stdout:12.2.12-835.ga6ab182.el7 2019-12-22T22:19:09.027 INFO:teuthology.packaging:The installed version of ceph is 12.2.12-835.ga6ab182.el7 2019-12-22T22:19:09.028 INFO:teuthology.task.install:The correct ceph version 12.2.12-835.ga6ab182 is installed. 2019-12-22T22:19:09.028 INFO:teuthology.orchestra.run.smithi107:> sudo lsb_release -is 2019-12-22T22:19:09.172 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS 2019-12-22T22:19:09.175 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-22T22:19:09.175 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2019-12-22T22:19:09.175 INFO:teuthology.packaging:ref: None 2019-12-22T22:19:09.175 INFO:teuthology.packaging:tag: None 2019-12-22T22:19:09.175 INFO:teuthology.packaging:branch: luminous 2019-12-22T22:19:09.175 INFO:teuthology.packaging:sha1: 99b509ffd4dd354919b08b38ba83b998fbfdc107 2019-12-22T22:19:09.176 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F7%2Fx86_64&ref=luminous 2019-12-22T22:19:09.586 INFO:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2019-12-22T22:19:09.724 INFO:teuthology.orchestra.run.smithi107.stdout:12.2.12-835.ga6ab182.el7 2019-12-22T22:19:09.724 INFO:teuthology.packaging:The installed version of ceph is 12.2.12-835.ga6ab182.el7 2019-12-22T22:19:09.724 INFO:teuthology.task.install:The correct ceph version 12.2.12-835.ga6ab182 is installed. 2019-12-22T22:19:09.725 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2019-12-22T22:19:09.726 INFO:teuthology.orchestra.run.smithi091:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2019-12-22T22:19:09.850 INFO:teuthology.orchestra.run.smithi043:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2019-12-22T22:19:09.971 INFO:teuthology.orchestra.run.smithi107:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2019-12-22T22:19:10.092 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2019-12-22T22:19:10.093 INFO:teuthology.orchestra.run.smithi091:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2019-12-22T22:19:10.212 INFO:teuthology.orchestra.run.smithi091:> sudo chmod a=rx -- /usr/bin/daemon-helper 2019-12-22T22:19:10.317 INFO:teuthology.orchestra.run.smithi043:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2019-12-22T22:19:10.442 INFO:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2019-12-22T22:19:10.560 INFO:teuthology.orchestra.run.smithi107:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2019-12-22T22:19:10.682 INFO:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2019-12-22T22:19:10.786 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2019-12-22T22:19:10.786 INFO:teuthology.orchestra.run.smithi091:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2019-12-22T22:19:10.910 INFO:teuthology.orchestra.run.smithi091:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2019-12-22T22:19:11.019 INFO:teuthology.orchestra.run.smithi043:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2019-12-22T22:19:11.137 INFO:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2019-12-22T22:19:11.225 INFO:teuthology.orchestra.run.smithi107:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2019-12-22T22:19:11.346 INFO:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2019-12-22T22:19:11.454 INFO:teuthology.run_tasks:Running task cephadm... 2019-12-22T22:19:11.636 INFO:tasks.cephadm:Config: {'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', '\\(MON_DOWN\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow requests'], 'mon_bind_msgr2': False, 'conf': {'client.2': {'debug objecter': 20, 'debug ms': 1, 'debug rados': 20}, 'global': {'ms inject delay max': 1, 'ms inject delay type': 'osd', 'osd_pool_default_min_size': 2, 'mon warn on msgr2 not enabled': False, 'ms inject delay probability': 0.005, 'mon client directed command retry': 5, 'ms inject socket failures': 2500, 'osd_pool_default_size': 2, 'ms inject internal delays': 0.002}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'debug ms': 1, 'mon osdmap full prune min': 15, 'mon osdmap full prune interval': 2, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'osd op queue cut off': 'debug_random', 'debug ms': 1, 'debug journal': 20, 'osd scrub max interval': 120, 'osd max backfills': 2, 'debug osd': 25, 'osd snap trim sleep': 0.5, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'filestore odsync write': True, 'osd op queue': 'debug_random', 'osd backoff on peering': True, 'osd scrub min interval': 60, 'debug filestore': 20, 'osd shutdown pgref assert': True}}, 'sha1': '99b509ffd4dd354919b08b38ba83b998fbfdc107'} 2019-12-22T22:19:11.637 INFO:tasks.cephadm:Cluster image is quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 2019-12-22T22:19:11.645 INFO:tasks.cephadm:Cluster fsid is 147630da-2509-11ea-8284-001a4aab830c 2019-12-22T22:19:11.645 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2019-12-22T22:19:11.645 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '[v1:172.21.15.91:6789]', 'mon.c': '[v1:172.21.15.43:6789]', 'mon.b': '[v1:172.21.15.107:6789]'} 2019-12-22T22:19:11.646 INFO:tasks.cephadm:Normalizing hostnames... 2019-12-22T22:19:11.646 INFO:teuthology.orchestra.run.smithi043:> sudo hostname $(hostname -s) 2019-12-22T22:19:11.771 INFO:teuthology.orchestra.run.smithi091:> sudo hostname $(hostname -s) 2019-12-22T22:19:11.889 INFO:teuthology.orchestra.run.smithi107:> sudo hostname $(hostname -s) 2019-12-22T22:19:12.008 INFO:tasks.cephadm:Downloading cephadm (repo git://git.ceph.com/ceph-ci.git ref 99b509ffd4dd354919b08b38ba83b998fbfdc107)... 2019-12-22T22:19:12.008 INFO:teuthology.orchestra.run.smithi043:> git archive --remote=git://git.ceph.com/ceph-ci.git 99b509ffd4dd354919b08b38ba83b998fbfdc107 src/cephadm/cephadm | tar -xO src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && test -s /home/ubuntu/cephtest/cephadm && chmod +x /home/ubuntu/cephtest/cephadm 2019-12-22T22:19:12.300 INFO:teuthology.orchestra.run.smithi091:> git archive --remote=git://git.ceph.com/ceph-ci.git 99b509ffd4dd354919b08b38ba83b998fbfdc107 src/cephadm/cephadm | tar -xO src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && test -s /home/ubuntu/cephtest/cephadm && chmod +x /home/ubuntu/cephtest/cephadm 2019-12-22T22:19:12.557 INFO:teuthology.orchestra.run.smithi107:> git archive --remote=git://git.ceph.com/ceph-ci.git 99b509ffd4dd354919b08b38ba83b998fbfdc107 src/cephadm/cephadm | tar -xO src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && test -s /home/ubuntu/cephtest/cephadm && chmod +x /home/ubuntu/cephtest/cephadm 2019-12-22T22:19:12.796 INFO:tasks.cephadm:First mon is mon.a on smithi091 2019-12-22T22:19:12.797 INFO:tasks.cephadm:others ['mon.a', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2019-12-22T22:19:12.797 INFO:tasks.cephadm:First mgr is y 2019-12-22T22:19:12.797 INFO:tasks.cephadm:Writing seed config... 2019-12-22T22:19:12.799 INFO:tasks.cephadm: override: [client.2] debug objecter = 20 2019-12-22T22:19:12.799 INFO:tasks.cephadm: override: [client.2] debug ms = 1 2019-12-22T22:19:12.800 INFO:tasks.cephadm: override: [client.2] debug rados = 20 2019-12-22T22:19:12.800 INFO:tasks.cephadm: override: [global] ms inject delay max = 1 2019-12-22T22:19:12.800 INFO:tasks.cephadm: override: [global] ms inject delay type = osd 2019-12-22T22:19:12.800 INFO:tasks.cephadm: override: [global] osd_pool_default_min_size = 2 2019-12-22T22:19:12.800 INFO:tasks.cephadm: override: [global] mon warn on msgr2 not enabled = False 2019-12-22T22:19:12.800 INFO:tasks.cephadm: override: [global] ms inject delay probability = 0.005 2019-12-22T22:19:12.801 INFO:tasks.cephadm: override: [global] mon client directed command retry = 5 2019-12-22T22:19:12.801 INFO:tasks.cephadm: override: [global] ms inject socket failures = 2500 2019-12-22T22:19:12.801 INFO:tasks.cephadm: override: [global] osd_pool_default_size = 2 2019-12-22T22:19:12.801 INFO:tasks.cephadm: override: [global] ms inject internal delays = 0.002 2019-12-22T22:19:12.801 INFO:tasks.cephadm: override: [mon] debug mon = 20 2019-12-22T22:19:12.801 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2019-12-22T22:19:12.802 INFO:tasks.cephadm: override: [mon] mon min osdmap epochs = 50 2019-12-22T22:19:12.802 INFO:tasks.cephadm: override: [mon] debug ms = 1 2019-12-22T22:19:12.802 INFO:tasks.cephadm: override: [mon] mon osdmap full prune min = 15 2019-12-22T22:19:12.802 INFO:tasks.cephadm: override: [mon] mon osdmap full prune interval = 2 2019-12-22T22:19:12.802 INFO:tasks.cephadm: override: [mon] mon osdmap full prune txsize = 2 2019-12-22T22:19:12.802 INFO:tasks.cephadm: override: [mon] paxos service trim min = 10 2019-12-22T22:19:12.803 INFO:tasks.cephadm: override: [osd] osd op queue cut off = debug_random 2019-12-22T22:19:12.803 INFO:tasks.cephadm: override: [osd] debug ms = 1 2019-12-22T22:19:12.803 INFO:tasks.cephadm: override: [osd] debug journal = 20 2019-12-22T22:19:12.803 INFO:tasks.cephadm: override: [osd] osd scrub max interval = 120 2019-12-22T22:19:12.803 INFO:tasks.cephadm: override: [osd] osd max backfills = 2 2019-12-22T22:19:12.803 INFO:tasks.cephadm: override: [osd] debug osd = 25 2019-12-22T22:19:12.804 INFO:tasks.cephadm: override: [osd] osd snap trim sleep = 0.5 2019-12-22T22:19:12.804 INFO:tasks.cephadm: override: [osd] osd debug verify cached snaps = True 2019-12-22T22:19:12.804 INFO:tasks.cephadm: override: [osd] osd debug verify missing on start = True 2019-12-22T22:19:12.804 INFO:tasks.cephadm: override: [osd] filestore odsync write = True 2019-12-22T22:19:12.805 INFO:tasks.cephadm: override: [osd] osd op queue = debug_random 2019-12-22T22:19:12.806 INFO:tasks.cephadm: override: [osd] osd backoff on peering = True 2019-12-22T22:19:12.806 INFO:tasks.cephadm: override: [osd] osd scrub min interval = 60 2019-12-22T22:19:12.806 INFO:tasks.cephadm: override: [osd] debug filestore = 20 2019-12-22T22:19:12.806 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2019-12-22T22:19:12.807 INFO:teuthology.orchestra.run.smithi091:> cat > /home/ubuntu/cephtest/seed.ceph.conf 2019-12-22T22:19:12.897 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 147630da-2509-11ea-8284-001a4aab830c ms inject delay max = 1 ms inject delay type = osd osd_pool_default_min_size = 2 mon warn on msgr2 not enabled = False ms inject delay probability = 0.005 mon client directed command retry = 5 ms inject socket failures = 2500 osd_pool_default_size = 2 ms inject internal delays = 0.002 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 120 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = True osd debug verify missing on start = True osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug journal = 20 osd max backfills = 2 debug osd = 25 osd snap trim sleep = 0.5 filestore odsync write = True osd backoff on peering = True osd scrub min interval = 60 debug filestore = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 debug mon = 20 debug paxos = 20 mon min osdmap epochs = 50 debug ms = 1 mon osdmap full prune min = 15 mon osdmap full prune interval = 2 mon osdmap full prune txsize = 2 paxos service trim min = 10 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client.2] debug objecter = 20 debug ms = 1 debug rados = 20 2019-12-22T22:19:12.897 INFO:tasks.cephadm:Bootstrapping... 2019-12-22T22:19:12.897 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 bootstrap --fsid 147630da-2509-11ea-8284-001a4aab830c --mon-id a --mgr-id y --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /home/ubuntu/cephtest/ceph.conf --output-keyring /home/ubuntu/cephtest/ceph.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --skip-ping-check --mon-addrv '[v1:172.21.15.91:6789]' && sudo chmod +r /home/ubuntu/cephtest/ceph.keyring 2019-12-22T22:19:13.171 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:root:Cluster fsid: 147630da-2509-11ea-8284-001a4aab830c 2019-12-22T22:19:13.172 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Pulling latest quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 container... 2019-12-22T22:19:42.517 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Extracting ceph user uid/gid from container image... 2019-12-22T22:19:51.927 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Creating initial keys... 2019-12-22T22:19:56.546 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Creating initial monmap... 2019-12-22T22:19:57.782 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Creating mon... 2019-12-22T22:19:59.509 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Waiting for mon to start... 2019-12-22T22:20:01.393 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Assimilating anything we can from ceph.conf... 2019-12-22T22:20:02.852 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Generating new minimal ceph.conf... 2019-12-22T22:20:04.152 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Restarting the monitor... 2019-12-22T22:20:05.006 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Creating mgr... 2019-12-22T22:20:05.390 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Creating crash agent... 2019-12-22T22:20:05.406 INFO:teuthology.orchestra.run.smithi091.stderr:Created symlink from /etc/systemd/system/ceph-147630da-2509-11ea-8284-001a4aab830c.target.wants/ceph-147630da-2509-11ea-8284-001a4aab830c-crash.service to /etc/systemd/system/ceph-147630da-2509-11ea-8284-001a4aab830c-crash.service. 2019-12-22T22:20:05.493 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Wrote keyring to /home/ubuntu/cephtest/ceph.keyring 2019-12-22T22:20:05.494 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Wrote config to /home/ubuntu/cephtest/ceph.conf 2019-12-22T22:20:05.494 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Waiting for mgr to start... 2019-12-22T22:20:07.292 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:mgr is still not available yet, waiting... 2019-12-22T22:20:09.592 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:mgr is still not available yet, waiting... 2019-12-22T22:20:11.998 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Enabling cephadm module... 2019-12-22T22:20:13.498 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Setting orchestrator backend to ssh... 2019-12-22T22:20:23.625 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Generating ssh key... 2019-12-22T22:20:36.558 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Wrote public SSH key to to /home/ubuntu/cephtest/ceph.pub 2019-12-22T22:20:36.558 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Adding key to root@localhost's authorized_keys... 2019-12-22T22:20:36.558 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Adding host smithi091... 2019-12-22T22:20:39.020 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Enabling the dashboard module... 2019-12-22T22:20:41.377 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Waiting for the module to be available... 2019-12-22T22:20:42.553 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Generating a dashboard self-signed certificate... 2019-12-22T22:20:46.569 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Creating initial admin user... 2019-12-22T22:20:48.355 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Fetching dashboard port number... 2019-12-22T22:20:49.708 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Ceph Dashboard is now available at: 2019-12-22T22:20:49.708 INFO:teuthology.orchestra.run.smithi091.stderr: 2019-12-22T22:20:49.709 INFO:teuthology.orchestra.run.smithi091.stderr: URL: https://smithi091.front.sepia.ceph.com:8443/ 2019-12-22T22:20:49.709 INFO:teuthology.orchestra.run.smithi091.stderr: User: admin 2019-12-22T22:20:49.709 INFO:teuthology.orchestra.run.smithi091.stderr: Password: clvssmswjg 2019-12-22T22:20:49.709 INFO:teuthology.orchestra.run.smithi091.stderr: 2019-12-22T22:20:49.710 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:You can access the Ceph CLI with: 2019-12-22T22:20:49.710 INFO:teuthology.orchestra.run.smithi091.stderr: 2019-12-22T22:20:49.710 INFO:teuthology.orchestra.run.smithi091.stderr: sudo /home/ubuntu/cephtest/cephadm shell --fsid 147630da-2509-11ea-8284-001a4aab830c -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring 2019-12-22T22:20:49.710 INFO:teuthology.orchestra.run.smithi091.stderr: 2019-12-22T22:20:49.711 INFO:teuthology.orchestra.run.smithi091.stderr:INFO:cephadm:Bootstrap complete. 2019-12-22T22:20:49.770 INFO:teuthology.orchestra.run.smithi091:mon.a> sudo /home/ubuntu/cephtest/cephadm logs -f --fsid 147630da-2509-11ea-8284-001a4aab830c --name mon.a 2019-12-22T22:20:49.776 INFO:teuthology.orchestra.run.smithi091:mgr.y> sudo /home/ubuntu/cephtest/cephadm logs -f --fsid 147630da-2509-11ea-8284-001a4aab830c --name mgr.y 2019-12-22T22:20:49.819 INFO:tasks.cephadm:Fetching config... 2019-12-22T22:20:49.916 DEBUG:teuthology.orchestra.remote:smithi091:/home/ubuntu/cephtest/ceph.conf is 147B 2019-12-22T22:20:49.975 INFO:tasks.cephadm:Fetching client.admin keyring... 2019-12-22T22:20:50.025 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.678+0000 7f9390f5e6c0 0 set uid:gid to 167:167 (ceph:ceph) 2019-12-22T22:20:50.025 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.678+0000 7f9390f5e6c0 0 ceph version 15.0.0-8807-g99b509f (99b509ffd4dd354919b08b38ba83b998fbfdc107) octopus (dev), process ceph-mon, pid 1 2019-12-22T22:20:50.027 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.678+0000 7f9390f5e6c0 0 pidfile_write: ignore empty --pid-file 2019-12-22T22:20:50.027 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.686+0000 7f9390f5e6c0 0 load: jerasure load: lrc load: isa 2019-12-22T22:20:50.028 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.686+0000 7f9390f5e6c0 0 set rocksdb option compression = kNoCompression 2019-12-22T22:20:50.028 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.686+0000 7f9390f5e6c0 0 set rocksdb option level_compaction_dynamic_level_bytes = true 2019-12-22T22:20:50.029 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.686+0000 7f9390f5e6c0 0 set rocksdb option write_buffer_size = 33554432 2019-12-22T22:20:50.029 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.686+0000 7f9390f5e6c0 0 set rocksdb option compression = kNoCompression 2019-12-22T22:20:50.029 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.686+0000 7f9390f5e6c0 0 set rocksdb option level_compaction_dynamic_level_bytes = true 2019-12-22T22:20:50.029 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.686+0000 7f9390f5e6c0 0 set rocksdb option write_buffer_size = 33554432 2019-12-22T22:20:50.029 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 1 rocksdb: do_open column families: [default] 2019-12-22T22:20:50.030 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: RocksDB version: 6.1.2 2019-12-22T22:20:50.030 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.030 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2019-12-22T22:20:50.030 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Compile date Dec 22 2019 2019-12-22T22:20:50.031 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: DB SUMMARY 2019-12-22T22:20:50.031 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.031 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: CURRENT file: CURRENT 2019-12-22T22:20:50.031 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.032 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: IDENTITY file: IDENTITY 2019-12-22T22:20:50.032 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.032 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 109 Bytes 2019-12-22T22:20:50.032 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.033 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000004.sst 2019-12-22T22:20:50.033 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.033 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000006.log size: 73197 ; 2019-12-22T22:20:50.034 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.034 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.error_if_exists: 0 2019-12-22T22:20:50.034 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.create_if_missing: 0 2019-12-22T22:20:50.034 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.paranoid_checks: 1 2019-12-22T22:20:50.035 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.env: 0x5619d5f76260 2019-12-22T22:20:50.035 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.info_log: 0x5619d7614480 2019-12-22T22:20:50.035 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.max_file_opening_threads: 16 2019-12-22T22:20:50.035 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.statistics: (nil) 2019-12-22T22:20:50.035 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.use_fsync: 0 2019-12-22T22:20:50.035 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.max_log_file_size: 0 2019-12-22T22:20:50.036 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.max_manifest_file_size: 1073741824 2019-12-22T22:20:50.036 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.log_file_time_to_roll: 0 2019-12-22T22:20:50.036 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.keep_log_file_num: 1000 2019-12-22T22:20:50.036 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.recycle_log_file_num: 0 2019-12-22T22:20:50.036 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.allow_fallocate: 1 2019-12-22T22:20:50.037 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.allow_mmap_reads: 0 2019-12-22T22:20:50.037 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.allow_mmap_writes: 0 2019-12-22T22:20:50.037 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.use_direct_reads: 0 2019-12-22T22:20:50.037 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2019-12-22T22:20:50.038 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.create_missing_column_families: 0 2019-12-22T22:20:50.051 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.db_log_dir: 2019-12-22T22:20:50.051 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2019-12-22T22:20:50.052 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.table_cache_numshardbits: 6 2019-12-22T22:20:50.052 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.max_subcompactions: 1 2019-12-22T22:20:50.052 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.max_background_flushes: -1 2019-12-22T22:20:50.052 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.WAL_ttl_seconds: 0 2019-12-22T22:20:50.053 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.WAL_size_limit_MB: 0 2019-12-22T22:20:50.053 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.manifest_preallocation_size: 4194304 2019-12-22T22:20:50.053 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.is_fd_close_on_exec: 1 2019-12-22T22:20:50.053 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.advise_random_on_open: 1 2019-12-22T22:20:50.054 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.db_write_buffer_size: 0 2019-12-22T22:20:50.054 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.write_buffer_manager: 0x5619d81bc990 2019-12-22T22:20:50.054 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.access_hint_on_compaction_start: 1 2019-12-22T22:20:50.054 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2019-12-22T22:20:50.054 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2019-12-22T22:20:50.055 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.use_adaptive_mutex: 0 2019-12-22T22:20:50.055 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.rate_limiter: (nil) 2019-12-22T22:20:50.055 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2019-12-22T22:20:50.055 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.wal_recovery_mode: 2 2019-12-22T22:20:50.055 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.enable_thread_tracking: 0 2019-12-22T22:20:50.056 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.enable_pipelined_write: 0 2019-12-22T22:20:50.056 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2019-12-22T22:20:50.056 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2019-12-22T22:20:50.076 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.write_thread_max_yield_usec: 100 2019-12-22T22:20:50.077 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2019-12-22T22:20:50.077 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.row_cache: None 2019-12-22T22:20:50.077 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.wal_filter: None 2019-12-22T22:20:50.077 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.avoid_flush_during_recovery: 0 2019-12-22T22:20:50.078 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.allow_ingest_behind: 0 2019-12-22T22:20:50.078 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.preserve_deletes: 0 2019-12-22T22:20:50.078 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.two_write_queues: 0 2019-12-22T22:20:50.078 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.manual_wal_flush: 0 2019-12-22T22:20:50.079 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.atomic_flush: 0 2019-12-22T22:20:50.079 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2019-12-22T22:20:50.079 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.max_background_jobs: 2 2019-12-22T22:20:50.079 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.max_background_compactions: -1 2019-12-22T22:20:50.079 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2019-12-22T22:20:50.080 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2019-12-22T22:20:50.080 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.delayed_write_rate : 16777216 2019-12-22T22:20:50.080 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.max_total_wal_size: 0 2019-12-22T22:20:50.080 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2019-12-22T22:20:50.081 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.stats_dump_period_sec: 600 2019-12-22T22:20:50.081 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.stats_persist_period_sec: 600 2019-12-22T22:20:50.081 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.stats_history_buffer_size: 1048576 2019-12-22T22:20:50.081 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.max_open_files: -1 2019-12-22T22:20:50.082 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.bytes_per_sync: 0 2019-12-22T22:20:50.082 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.wal_bytes_per_sync: 0 2019-12-22T22:20:50.082 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Options.compaction_readahead_size: 0 2019-12-22T22:20:50.082 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Compression algorithms supported: 2019-12-22T22:20:50.082 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: kZSTDNotFinalCompression supported: 0 2019-12-22T22:20:50.082 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: kZSTD supported: 0 2019-12-22T22:20:50.083 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: kXpressCompression supported: 0 2019-12-22T22:20:50.083 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: kLZ4HCCompression supported: 1 2019-12-22T22:20:50.083 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: kLZ4Compression supported: 1 2019-12-22T22:20:50.083 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: kBZip2Compression supported: 0 2019-12-22T22:20:50.084 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: kZlibCompression supported: 1 2019-12-22T22:20:50.084 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: kSnappyCompression supported: 1 2019-12-22T22:20:50.084 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.687+0000 7f9390f5e6c0 4 rocksdb: Fast CRC32 supported: Supported on x86 2019-12-22T22:20:50.084 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: [db/version_set.cc:3543] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 2019-12-22T22:20:50.085 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.085 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: [db/column_family.cc:477] --------------- Options for column family [default]: 2019-12-22T22:20:50.085 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.085 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2019-12-22T22:20:50.085 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.merge_operator: 2019-12-22T22:20:50.086 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.compaction_filter: None 2019-12-22T22:20:50.086 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.compaction_filter_factory: None 2019-12-22T22:20:50.086 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.memtable_factory: SkipListFactory 2019-12-22T22:20:50.086 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.table_factory: BlockBasedTable 2019-12-22T22:20:50.087 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5619d75300c8) 2019-12-22T22:20:50.087 INFO:ceph.mon.a.smithi091.stdout: cache_index_and_filter_blocks: 1 2019-12-22T22:20:50.087 INFO:ceph.mon.a.smithi091.stdout: cache_index_and_filter_blocks_with_high_priority: 1 2019-12-22T22:20:50.087 INFO:ceph.mon.a.smithi091.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2019-12-22T22:20:50.088 INFO:ceph.mon.a.smithi091.stdout: pin_top_level_index_and_filter: 1 2019-12-22T22:20:50.088 INFO:ceph.mon.a.smithi091.stdout: index_type: 0 2019-12-22T22:20:50.088 INFO:ceph.mon.a.smithi091.stdout: data_block_index_type: 0 2019-12-22T22:20:50.088 INFO:ceph.mon.a.smithi091.stdout: data_block_hash_table_util_ratio: 0.750000 2019-12-22T22:20:50.089 INFO:ceph.mon.a.smithi091.stdout: hash_index_allow_collision: 1 2019-12-22T22:20:50.089 INFO:ceph.mon.a.smithi091.stdout: checksum: 1 2019-12-22T22:20:50.089 INFO:ceph.mon.a.smithi091.stdout: no_block_cache: 0 2019-12-22T22:20:50.089 INFO:ceph.mon.a.smithi091.stdout: block_cache: 0x5619d756a210 2019-12-22T22:20:50.089 INFO:ceph.mon.a.smithi091.stdout: block_cache_name: BinnedLRUCache 2019-12-22T22:20:50.090 INFO:ceph.mon.a.smithi091.stdout: block_cache_options: 2019-12-22T22:20:50.090 INFO:ceph.mon.a.smithi091.stdout: capacity : 536870912 2019-12-22T22:20:50.090 INFO:ceph.mon.a.smithi091.stdout: num_shard_bits : 4 2019-12-22T22:20:50.090 INFO:ceph.mon.a.smithi091.stdout: strict_capacity_limit : 0 2019-12-22T22:20:50.091 INFO:ceph.mon.a.smithi091.stdout: high_pri_pool_ratio: 0.000 2019-12-22T22:20:50.091 INFO:ceph.mon.a.smithi091.stdout: block_cache_compressed: (nil) 2019-12-22T22:20:50.091 INFO:ceph.mon.a.smithi091.stdout: persistent_cache: (nil) 2019-12-22T22:20:50.091 INFO:ceph.mon.a.smithi091.stdout: block_size: 4096 2019-12-22T22:20:50.091 INFO:ceph.mon.a.smithi091.stdout: block_size_deviation: 10 2019-12-22T22:20:50.091 INFO:ceph.mon.a.smithi091.stdout: block_restart_interval: 16 2019-12-22T22:20:50.092 INFO:ceph.mon.a.smithi091.stdout: index_block_restart_interval: 1 2019-12-22T22:20:50.092 INFO:ceph.mon.a.smithi091.stdout: metadata_block_size: 4096 2019-12-22T22:20:50.092 INFO:ceph.mon.a.smithi091.stdout: partition_filters: 0 2019-12-22T22:20:50.092 INFO:ceph.mon.a.smithi091.stdout: use_delta_encoding: 1 2019-12-22T22:20:50.093 INFO:ceph.mon.a.smithi091.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2019-12-22T22:20:50.093 INFO:ceph.mon.a.smithi091.stdout: whole_key_filtering: 1 2019-12-22T22:20:50.093 INFO:ceph.mon.a.smithi091.stdout: verify_compression: 0 2019-12-22T22:20:50.093 INFO:ceph.mon.a.smithi091.stdout: read_amp_bytes_per_bit: 0 2019-12-22T22:20:50.093 INFO:ceph.mon.a.smithi091.stdout: format_version: 2 2019-12-22T22:20:50.094 INFO:ceph.mon.a.smithi091.stdout: enable_index_compression: 1 2019-12-22T22:20:50.094 INFO:ceph.mon.a.smithi091.stdout: block_align: 0 2019-12-22T22:20:50.094 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.094 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.write_buffer_size: 33554432 2019-12-22T22:20:50.095 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.max_write_buffer_number: 2 2019-12-22T22:20:50.095 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.compression: NoCompression 2019-12-22T22:20:50.095 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.bottommost_compression: Disabled 2019-12-22T22:20:50.095 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.prefix_extractor: nullptr 2019-12-22T22:20:50.095 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2019-12-22T22:20:50.096 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.num_levels: 7 2019-12-22T22:20:50.096 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2019-12-22T22:20:50.096 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2019-12-22T22:20:50.096 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2019-12-22T22:20:50.096 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2019-12-22T22:20:50.097 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2019-12-22T22:20:50.097 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2019-12-22T22:20:50.097 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2019-12-22T22:20:50.097 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2019-12-22T22:20:50.098 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.compression_opts.window_bits: -14 2019-12-22T22:20:50.098 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.compression_opts.level: 32767 2019-12-22T22:20:50.098 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.compression_opts.strategy: 0 2019-12-22T22:20:50.098 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2019-12-22T22:20:50.098 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2019-12-22T22:20:50.099 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.compression_opts.enabled: false 2019-12-22T22:20:50.099 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2019-12-22T22:20:50.099 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2019-12-22T22:20:50.099 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.level0_stop_writes_trigger: 36 2019-12-22T22:20:50.100 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.target_file_size_base: 67108864 2019-12-22T22:20:50.100 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.target_file_size_multiplier: 1 2019-12-22T22:20:50.100 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2019-12-22T22:20:50.100 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2019-12-22T22:20:50.100 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2019-12-22T22:20:50.101 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2019-12-22T22:20:50.101 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2019-12-22T22:20:50.101 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2019-12-22T22:20:50.101 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2019-12-22T22:20:50.102 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2019-12-22T22:20:50.102 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2019-12-22T22:20:50.102 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2019-12-22T22:20:50.102 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2019-12-22T22:20:50.103 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.max_compaction_bytes: 1677721600 2019-12-22T22:20:50.103 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.arena_block_size: 4194304 2019-12-22T22:20:50.103 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2019-12-22T22:20:50.103 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2019-12-22T22:20:50.104 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2019-12-22T22:20:50.104 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.disable_auto_compactions: 0 2019-12-22T22:20:50.104 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2019-12-22T22:20:50.104 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2019-12-22T22:20:50.104 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2019-12-22T22:20:50.104 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2019-12-22T22:20:50.104 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2019-12-22T22:20:50.105 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2019-12-22T22:20:50.105 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2019-12-22T22:20:50.105 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2019-12-22T22:20:50.105 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2019-12-22T22:20:50.105 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2019-12-22T22:20:50.105 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.table_properties_collectors: 2019-12-22T22:20:50.105 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.inplace_update_support: 0 2019-12-22T22:20:50.106 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.inplace_update_num_locks: 10000 2019-12-22T22:20:50.106 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2019-12-22T22:20:50.106 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.memtable_whole_key_filtering: 0 2019-12-22T22:20:50.106 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.memtable_huge_page_size: 0 2019-12-22T22:20:50.106 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.bloom_locality: 0 2019-12-22T22:20:50.106 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.max_successive_merges: 0 2019-12-22T22:20:50.107 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.optimize_filters_for_hits: 0 2019-12-22T22:20:50.107 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.paranoid_file_checks: 0 2019-12-22T22:20:50.107 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.force_consistency_checks: 0 2019-12-22T22:20:50.107 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.report_bg_io_stats: 0 2019-12-22T22:20:50.108 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.688+0000 7f9390f5e6c0 4 rocksdb: Options.ttl: 0 2019-12-22T22:20:50.108 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.689+0000 7f9390f5e6c0 4 rocksdb: [db/version_set.cc:3757] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 5, log_number is 4,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2019-12-22T22:20:50.108 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.108 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.689+0000 7f9390f5e6c0 4 rocksdb: [db/version_set.cc:3766] Column family [default] (ID 0), log number is 4 2019-12-22T22:20:50.108 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.108 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.689+0000 7f9390f5e6c0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1577053205690453, "job": 1, "event": "recovery_started", "log_files": [6]} 2019-12-22T22:20:50.109 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.689+0000 7f9390f5e6c0 4 rocksdb: [db/db_impl_open.cc:583] Recovering log #6 mode 2 2019-12-22T22:20:50.109 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.690+0000 7f9390f5e6c0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1577053205691874, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 7, "file_size": 69065, "table_properties": {"data_size": 67365, "index_size": 305, "filter_size": 581, "raw_key_size": 11202, "raw_average_key_size": 49, "raw_value_size": 61241, "raw_average_value_size": 272, "num_data_blocks": 10, "num_entries": 225, "filter_policy_name": "rocksdb.BuiltinBloomFilter"}} 2019-12-22T22:20:50.109 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.690+0000 7f9390f5e6c0 4 rocksdb: [db/version_set.cc:3036] Creating manifest 8 2019-12-22T22:20:50.109 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.109 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.691+0000 7f9390f5e6c0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1577053205692488, "job": 1, "event": "recovery_finished"} 2019-12-22T22:20:50.109 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.693+0000 7f9390f5e6c0 4 rocksdb: DB pointer 0x5619d819f200 2019-12-22T22:20:50.110 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.693+0000 7f9378091700 4 rocksdb: [db/db_impl.cc:777] ------- DUMPING STATS ------- 2019-12-22T22:20:50.110 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.693+0000 7f9378091700 4 rocksdb: [db/db_impl.cc:778] 2019-12-22T22:20:50.110 INFO:ceph.mon.a.smithi091.stdout:** DB Stats ** 2019-12-22T22:20:50.110 INFO:ceph.mon.a.smithi091.stdout:Uptime(secs): 0.0 total, 0.0 interval 2019-12-22T22:20:50.110 INFO:ceph.mon.a.smithi091.stdout:Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2019-12-22T22:20:50.110 INFO:ceph.mon.a.smithi091.stdout:Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2019-12-22T22:20:50.110 INFO:ceph.mon.a.smithi091.stdout:Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2019-12-22T22:20:50.111 INFO:ceph.mon.a.smithi091.stdout:Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2019-12-22T22:20:50.111 INFO:ceph.mon.a.smithi091.stdout:Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2019-12-22T22:20:50.111 INFO:ceph.mon.a.smithi091.stdout:Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2019-12-22T22:20:50.111 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.111 INFO:ceph.mon.a.smithi091.stdout:** Compaction Stats [default] ** 2019-12-22T22:20:50.111 INFO:ceph.mon.a.smithi091.stdout:Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2019-12-22T22:20:50.111 INFO:ceph.mon.a.smithi091.stdout:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2019-12-22T22:20:50.111 INFO:ceph.mon.a.smithi091.stdout: L0 2/0 69.16 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 70.1 0.00 0.00 1 0.001 0 0 2019-12-22T22:20:50.112 INFO:ceph.mon.a.smithi091.stdout: Sum 2/0 69.16 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 70.1 0.00 0.00 1 0.001 0 0 2019-12-22T22:20:50.112 INFO:ceph.mon.a.smithi091.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 70.1 0.00 0.00 1 0.001 0 0 2019-12-22T22:20:50.112 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.112 INFO:ceph.mon.a.smithi091.stdout:** Compaction Stats [default] ** 2019-12-22T22:20:50.112 INFO:ceph.mon.a.smithi091.stdout:Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2019-12-22T22:20:50.112 INFO:ceph.mon.a.smithi091.stdout:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2019-12-22T22:20:50.113 INFO:ceph.mon.a.smithi091.stdout:User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 70.1 0.00 0.00 1 0.001 0 0 2019-12-22T22:20:50.113 INFO:ceph.mon.a.smithi091.stdout:Uptime(secs): 0.0 total, 0.0 interval 2019-12-22T22:20:50.113 INFO:ceph.mon.a.smithi091.stdout:Flush(GB): cumulative 0.000, interval 0.000 2019-12-22T22:20:50.113 INFO:ceph.mon.a.smithi091.stdout:AddFile(GB): cumulative 0.000, interval 0.000 2019-12-22T22:20:50.113 INFO:ceph.mon.a.smithi091.stdout:AddFile(Total Files): cumulative 0, interval 0 2019-12-22T22:20:50.114 INFO:ceph.mon.a.smithi091.stdout:AddFile(L0 Files): cumulative 0, interval 0 2019-12-22T22:20:50.114 INFO:ceph.mon.a.smithi091.stdout:AddFile(Keys): cumulative 0, interval 0 2019-12-22T22:20:50.114 INFO:ceph.mon.a.smithi091.stdout:Cumulative compaction: 0.00 GB write, 13.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2019-12-22T22:20:50.114 INFO:ceph.mon.a.smithi091.stdout:Interval compaction: 0.00 GB write, 13.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2019-12-22T22:20:50.114 INFO:ceph.mon.a.smithi091.stdout:Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2019-12-22T22:20:50.114 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.115 INFO:ceph.mon.a.smithi091.stdout:** File Read Latency Histogram By Level [default] ** 2019-12-22T22:20:50.115 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.115 INFO:ceph.mon.a.smithi091.stdout:** Compaction Stats [default] ** 2019-12-22T22:20:50.115 INFO:ceph.mon.a.smithi091.stdout:Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2019-12-22T22:20:50.115 INFO:ceph.mon.a.smithi091.stdout:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2019-12-22T22:20:50.116 INFO:ceph.mon.a.smithi091.stdout: L0 2/0 69.16 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 70.1 0.00 0.00 1 0.001 0 0 2019-12-22T22:20:50.116 INFO:ceph.mon.a.smithi091.stdout: Sum 2/0 69.16 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 70.1 0.00 0.00 1 0.001 0 0 2019-12-22T22:20:50.116 INFO:ceph.mon.a.smithi091.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2019-12-22T22:20:50.116 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.116 INFO:ceph.mon.a.smithi091.stdout:** Compaction Stats [default] ** 2019-12-22T22:20:50.116 INFO:ceph.mon.a.smithi091.stdout:Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2019-12-22T22:20:50.116 INFO:ceph.mon.a.smithi091.stdout:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2019-12-22T22:20:50.117 INFO:ceph.mon.a.smithi091.stdout:User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 70.1 0.00 0.00 1 0.001 0 0 2019-12-22T22:20:50.117 INFO:ceph.mon.a.smithi091.stdout:Uptime(secs): 0.0 total, 0.0 interval 2019-12-22T22:20:50.117 INFO:ceph.mon.a.smithi091.stdout:Flush(GB): cumulative 0.000, interval 0.000 2019-12-22T22:20:50.117 INFO:ceph.mon.a.smithi091.stdout:AddFile(GB): cumulative 0.000, interval 0.000 2019-12-22T22:20:50.117 INFO:ceph.mon.a.smithi091.stdout:AddFile(Total Files): cumulative 0, interval 0 2019-12-22T22:20:50.117 INFO:ceph.mon.a.smithi091.stdout:AddFile(L0 Files): cumulative 0, interval 0 2019-12-22T22:20:50.118 INFO:ceph.mon.a.smithi091.stdout:AddFile(Keys): cumulative 0, interval 0 2019-12-22T22:20:50.118 INFO:ceph.mon.a.smithi091.stdout:Cumulative compaction: 0.00 GB write, 12.86 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2019-12-22T22:20:50.118 INFO:ceph.mon.a.smithi091.stdout:Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2019-12-22T22:20:50.118 INFO:ceph.mon.a.smithi091.stdout:Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2019-12-22T22:20:50.118 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.118 INFO:ceph.mon.a.smithi091.stdout:** File Read Latency Histogram By Level [default] ** 2019-12-22T22:20:50.118 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.118 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.694+0000 7f9390f5e6c0 0 starting mon.a rank 0 at public addrs v1:172.21.15.91:6789/0 at bind addrs v1:172.21.15.91:6789/0 mon_data /var/lib/ceph/mon/ceph-a fsid 147630da-2509-11ea-8284-001a4aab830c 2019-12-22T22:20:50.119 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.695+0000 7f9390f5e6c0 1 mon.a@-1(???) e1 preinit fsid 147630da-2509-11ea-8284-001a4aab830c 2019-12-22T22:20:50.119 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.695+0000 7f9390f5e6c0 0 mon.a@-1(???).mds e1 new map 2019-12-22T22:20:50.119 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.695+0000 7f9390f5e6c0 0 mon.a@-1(???).mds e1 print_map 2019-12-22T22:20:50.119 INFO:ceph.mon.a.smithi091.stdout:e1 2019-12-22T22:20:50.120 INFO:ceph.mon.a.smithi091.stdout:enable_multiple, ever_enabled_multiple: 0,0 2019-12-22T22:20:50.120 INFO:ceph.mon.a.smithi091.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2019-12-22T22:20:50.120 INFO:ceph.mon.a.smithi091.stdout:legacy client fscid: -1 2019-12-22T22:20:50.120 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.121 INFO:ceph.mon.a.smithi091.stdout:No filesystems configured 2019-12-22T22:20:50.121 INFO:ceph.mon.a.smithi091.stdout: 2019-12-22T22:20:50.121 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.696+0000 7f9390f5e6c0 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2019-12-22T22:20:50.122 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.696+0000 7f9390f5e6c0 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2019-12-22T22:20:50.122 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.696+0000 7f9390f5e6c0 0 mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2019-12-22T22:20:50.122 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.696+0000 7f9390f5e6c0 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2019-12-22T22:20:50.122 INFO:ceph.mon.a.smithi091.stdout:2019-12-22T22:20:05.696+0000 7f9390f5e6c0 1 mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2019-12-22T22:20:50.124 INFO:ceph.mgr.y.smithi091.stdout:ignoring --setuser ceph since I am not root 2019-12-22T22:20:50.124 INFO:ceph.mgr.y.smithi091.stdout:ignoring --setgroup ceph since I am not root 2019-12-22T22:20:50.125 INFO:ceph.mgr.y.smithi091.stdout:Generating public/private rsa key pair. 2019-12-22T22:20:50.125 INFO:ceph.mgr.y.smithi091.stdout:Your identification has been saved in /tmp/tmpuqmacx3l/key. 2019-12-22T22:20:50.125 INFO:ceph.mgr.y.smithi091.stdout:Your public key has been saved in /tmp/tmpuqmacx3l/key.pub. 2019-12-22T22:20:50.125 INFO:ceph.mgr.y.smithi091.stdout:The key fingerprint is: 2019-12-22T22:20:50.126 INFO:ceph.mgr.y.smithi091.stdout:SHA256:VG4Suuim9mqa4NPiIEUw7Ca1NyMKH6+oxmuSNUYIXzk ceph-147630da-2509-11ea-8284-001a4aab830c 2019-12-22T22:20:50.126 INFO:ceph.mgr.y.smithi091.stdout:The key's randomart image is: 2019-12-22T22:20:50.126 INFO:ceph.mgr.y.smithi091.stdout:+---[RSA 2048]----+ 2019-12-22T22:20:50.126 INFO:ceph.mgr.y.smithi091.stdout:|+ . . . | 2019-12-22T22:20:50.127 INFO:ceph.mgr.y.smithi091.stdout:|.+. E . + | 2019-12-22T22:20:50.127 INFO:ceph.mgr.y.smithi091.stdout:|o+.o .. o o | 2019-12-22T22:20:50.127 INFO:ceph.mgr.y.smithi091.stdout:|++* +. o o | 2019-12-22T22:20:50.127 INFO:ceph.mgr.y.smithi091.stdout:|++.=.o. S | 2019-12-22T22:20:50.127 INFO:ceph.mgr.y.smithi091.stdout:|..=.. | 2019-12-22T22:20:50.128 INFO:ceph.mgr.y.smithi091.stdout:|==.oo | 2019-12-22T22:20:50.128 INFO:ceph.mgr.y.smithi091.stdout:|OB++ | 2019-12-22T22:20:50.128 INFO:ceph.mgr.y.smithi091.stdout:|XB*o. | 2019-12-22T22:20:50.128 INFO:ceph.mgr.y.smithi091.stdout:+----[SHA256]-----+ 2019-12-22T22:20:50.128 INFO:ceph.mgr.y.smithi091.stdout:ignoring --setuser ceph since I am not root 2019-12-22T22:20:50.129 INFO:ceph.mgr.y.smithi091.stdout:ignoring --setgroup ceph since I am not root 2019-12-22T22:20:50.132 DEBUG:teuthology.orchestra.remote:smithi091:/home/ubuntu/cephtest/ceph.keyring is 63B 2019-12-22T22:20:50.207 INFO:tasks.cephadm:Fetching mon keyring... 2019-12-22T22:20:50.208 INFO:teuthology.orchestra.run.smithi091:> mktemp 2019-12-22T22:20:50.281 INFO:teuthology.orchestra.run.smithi091.stdout:/tmp/tmp.yCzYNMV9u5 2019-12-22T22:20:50.282 INFO:teuthology.orchestra.run.smithi091:> sudo cp /var/lib/ceph/147630da-2509-11ea-8284-001a4aab830c/mon.a/keyring /tmp/tmp.yCzYNMV9u5 2019-12-22T22:20:50.442 INFO:teuthology.orchestra.run.smithi091:> sudo chmod 0666 /tmp/tmp.yCzYNMV9u5 2019-12-22T22:20:50.661 DEBUG:teuthology.orchestra.remote:smithi091:/tmp/tmp.yCzYNMV9u5 is 77B 2019-12-22T22:20:50.743 INFO:teuthology.orchestra.run.smithi091:> rm -fr /tmp/tmp.yCzYNMV9u5 2019-12-22T22:20:50.826 INFO:tasks.cephadm:Fetching pub ssh key... 2019-12-22T22:20:50.961 DEBUG:teuthology.orchestra.remote:smithi091:/home/ubuntu/cephtest/ceph.pub is 423B 2019-12-22T22:20:51.056 INFO:tasks.cephadm:Installing pub ssh key for root users... 2019-12-22T22:20:51.057 INFO:teuthology.orchestra.run.smithi043:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZKBEXw2T+G4bWDqi3KPHXyQTVgpIS+q0TrZEi/96DAjy+UKlmiFxO+fol6wdPGq3jXi8y0ROp/770/IkuPvbI34Nb2h3P15PdcWNkYMjBIxl+T4mwf5VPVpgCtcUFPQLXAlx5Y4kCZTDuJlRLiU/1jRESvJhq9JglicnX2DlVa7xl1lp4LnKHD7Ixm+9q68pSMgNTM/uMDXahK1RqbcB1ZW/OwGFQHQIK6klpeCmA72rEElEdbOQKZl6InDlDg8W4gCdbVsAB9MF1/MJDDAS79SqTugsmBedT3yye5tXB0AW1PXfkajWvd2oqvHK91CtJBTtUw6OQiJIZsooKJxj ceph-147630da-2509-11ea-8284-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2019-12-22T22:20:51.229 INFO:teuthology.orchestra.run.smithi043.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZKBEXw2T+G4bWDqi3KPHXyQTVgpIS+q0TrZEi/96DAjy+UKlmiFxO+fol6wdPGq3jXi8y0ROp/770/IkuPvbI34Nb2h3P15PdcWNkYMjBIxl+T4mwf5VPVpgCtcUFPQLXAlx5Y4kCZTDuJlRLiU/1jRESvJhq9JglicnX2DlVa7xl1lp4LnKHD7Ixm+9q68pSMgNTM/uMDXahK1RqbcB1ZW/OwGFQHQIK6klpeCmA72rEElEdbOQKZl6InDlDg8W4gCdbVsAB9MF1/MJDDAS79SqTugsmBedT3yye5tXB0AW1PXfkajWvd2oqvHK91CtJBTtUw6OQiJIZsooKJxj ceph-147630da-2509-11ea-8284-001a4aab830c 2019-12-22T22:20:51.261 INFO:teuthology.orchestra.run.smithi091:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZKBEXw2T+G4bWDqi3KPHXyQTVgpIS+q0TrZEi/96DAjy+UKlmiFxO+fol6wdPGq3jXi8y0ROp/770/IkuPvbI34Nb2h3P15PdcWNkYMjBIxl+T4mwf5VPVpgCtcUFPQLXAlx5Y4kCZTDuJlRLiU/1jRESvJhq9JglicnX2DlVa7xl1lp4LnKHD7Ixm+9q68pSMgNTM/uMDXahK1RqbcB1ZW/OwGFQHQIK6klpeCmA72rEElEdbOQKZl6InDlDg8W4gCdbVsAB9MF1/MJDDAS79SqTugsmBedT3yye5tXB0AW1PXfkajWvd2oqvHK91CtJBTtUw6OQiJIZsooKJxj ceph-147630da-2509-11ea-8284-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2019-12-22T22:20:51.415 INFO:teuthology.orchestra.run.smithi091.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZKBEXw2T+G4bWDqi3KPHXyQTVgpIS+q0TrZEi/96DAjy+UKlmiFxO+fol6wdPGq3jXi8y0ROp/770/IkuPvbI34Nb2h3P15PdcWNkYMjBIxl+T4mwf5VPVpgCtcUFPQLXAlx5Y4kCZTDuJlRLiU/1jRESvJhq9JglicnX2DlVa7xl1lp4LnKHD7Ixm+9q68pSMgNTM/uMDXahK1RqbcB1ZW/OwGFQHQIK6klpeCmA72rEElEdbOQKZl6InDlDg8W4gCdbVsAB9MF1/MJDDAS79SqTugsmBedT3yye5tXB0AW1PXfkajWvd2oqvHK91CtJBTtUw6OQiJIZsooKJxj ceph-147630da-2509-11ea-8284-001a4aab830c 2019-12-22T22:20:51.447 INFO:teuthology.orchestra.run.smithi107:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZKBEXw2T+G4bWDqi3KPHXyQTVgpIS+q0TrZEi/96DAjy+UKlmiFxO+fol6wdPGq3jXi8y0ROp/770/IkuPvbI34Nb2h3P15PdcWNkYMjBIxl+T4mwf5VPVpgCtcUFPQLXAlx5Y4kCZTDuJlRLiU/1jRESvJhq9JglicnX2DlVa7xl1lp4LnKHD7Ixm+9q68pSMgNTM/uMDXahK1RqbcB1ZW/OwGFQHQIK6klpeCmA72rEElEdbOQKZl6InDlDg8W4gCdbVsAB9MF1/MJDDAS79SqTugsmBedT3yye5tXB0AW1PXfkajWvd2oqvHK91CtJBTtUw6OQiJIZsooKJxj ceph-147630da-2509-11ea-8284-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2019-12-22T22:20:51.622 INFO:teuthology.orchestra.run.smithi107.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZKBEXw2T+G4bWDqi3KPHXyQTVgpIS+q0TrZEi/96DAjy+UKlmiFxO+fol6wdPGq3jXi8y0ROp/770/IkuPvbI34Nb2h3P15PdcWNkYMjBIxl+T4mwf5VPVpgCtcUFPQLXAlx5Y4kCZTDuJlRLiU/1jRESvJhq9JglicnX2DlVa7xl1lp4LnKHD7Ixm+9q68pSMgNTM/uMDXahK1RqbcB1ZW/OwGFQHQIK6klpeCmA72rEElEdbOQKZl6InDlDg8W4gCdbVsAB9MF1/MJDDAS79SqTugsmBedT3yye5tXB0AW1PXfkajWvd2oqvHK91CtJBTtUw6OQiJIZsooKJxj ceph-147630da-2509-11ea-8284-001a4aab830c 2019-12-22T22:20:51.648 INFO:tasks.cephadm:Writing conf and keyring to smithi043 2019-12-22T22:20:51.648 INFO:teuthology.orchestra.run.smithi043:> cat > /home/ubuntu/cephtest/ceph.conf 2019-12-22T22:20:51.742 INFO:teuthology.orchestra.run.smithi043:> cat > /home/ubuntu/cephtest/ceph.keyring 2019-12-22T22:20:51.816 INFO:tasks.cephadm:Adding host smithi043 to orchestrator... 2019-12-22T22:20:51.817 INFO:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph orchestrator host add smithi043 2019-12-22T22:21:32.206 INFO:teuthology.orchestra.run.smithi043.stdout:Added host 'smithi043' 2019-12-22T22:21:32.905 INFO:tasks.cephadm:Writing conf and keyring to smithi107 2019-12-22T22:21:32.905 INFO:teuthology.orchestra.run.smithi107:> cat > /home/ubuntu/cephtest/ceph.conf 2019-12-22T22:21:32.997 INFO:teuthology.orchestra.run.smithi107:> cat > /home/ubuntu/cephtest/ceph.keyring 2019-12-22T22:21:33.081 INFO:tasks.cephadm:Adding host smithi107 to orchestrator... 2019-12-22T22:21:33.081 INFO:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph orchestrator host add smithi107 2019-12-22T22:22:12.565 INFO:teuthology.orchestra.run.smithi107.stdout:Added host 'smithi107' 2019-12-22T22:22:13.325 INFO:tasks.cephadm:Adding mon.c on smithi043 2019-12-22T22:22:13.325 INFO:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph orchestrator mon update 2 'smithi043:[v1:172.21.15.43:6789]=c' 2019-12-22T22:22:15.821 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi043,172.21.15.43' (ECDSA) to the list of known hosts. 2019-12-22T22:22:16.571 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi091,172.21.15.91' (ECDSA) to the list of known hosts. 2019-12-22T22:22:17.573 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi107,172.21.15.107' (ECDSA) to the list of known hosts. 2019-12-22T22:22:18.572 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi043,172.21.15.43' (ECDSA) to the list of known hosts. 2019-12-22T22:22:22.546 INFO:teuthology.orchestra.run.smithi043.stdout:["(Re)deployed mon.c on host 'smithi043'"] 2019-12-22T22:22:23.266 INFO:teuthology.orchestra.run.smithi043:mon.c> sudo /home/ubuntu/cephtest/cephadm logs -f --fsid 147630da-2509-11ea-8284-001a4aab830c --name mon.c 2019-12-22T22:22:23.272 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2019-12-22T22:22:23.273 INFO:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph mon dump -f json 2019-12-22T22:22:23.590 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.697+0000 7f5c12ae36c0 0 set uid:gid to 167:167 (ceph:ceph) 2019-12-22T22:22:23.590 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.697+0000 7f5c12ae36c0 0 ceph version 15.0.0-8807-g99b509f (99b509ffd4dd354919b08b38ba83b998fbfdc107) octopus (dev), process ceph-mon, pid 1 2019-12-22T22:22:23.591 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.697+0000 7f5c12ae36c0 0 pidfile_write: ignore empty --pid-file 2019-12-22T22:22:23.591 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.705+0000 7f5c12ae36c0 0 load: jerasure load: lrc load: isa 2019-12-22T22:22:23.591 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.705+0000 7f5c12ae36c0 0 set rocksdb option compression = kNoCompression 2019-12-22T22:22:23.591 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.705+0000 7f5c12ae36c0 0 set rocksdb option level_compaction_dynamic_level_bytes = true 2019-12-22T22:22:23.591 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.705+0000 7f5c12ae36c0 0 set rocksdb option write_buffer_size = 33554432 2019-12-22T22:22:23.592 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.705+0000 7f5c12ae36c0 0 set rocksdb option compression = kNoCompression 2019-12-22T22:22:23.592 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.705+0000 7f5c12ae36c0 0 set rocksdb option level_compaction_dynamic_level_bytes = true 2019-12-22T22:22:23.592 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.705+0000 7f5c12ae36c0 0 set rocksdb option write_buffer_size = 33554432 2019-12-22T22:22:23.592 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 1 rocksdb: do_open column families: [default] 2019-12-22T22:22:23.592 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: RocksDB version: 6.1.2 2019-12-22T22:22:23.593 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.593 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2019-12-22T22:22:23.594 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Compile date Dec 22 2019 2019-12-22T22:22:23.594 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: DB SUMMARY 2019-12-22T22:22:23.594 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.594 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: CURRENT file: CURRENT 2019-12-22T22:22:23.595 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.595 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: IDENTITY file: IDENTITY 2019-12-22T22:22:23.595 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.595 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: MANIFEST file: MANIFEST-000001 size: 13 Bytes 2019-12-22T22:22:23.595 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.595 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2019-12-22T22:22:23.595 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.596 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000003.log size: 454 ; 2019-12-22T22:22:23.596 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.596 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.error_if_exists: 0 2019-12-22T22:22:23.596 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.create_if_missing: 0 2019-12-22T22:22:23.596 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.paranoid_checks: 1 2019-12-22T22:22:23.597 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.env: 0x555aab4fe260 2019-12-22T22:22:23.597 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.info_log: 0x555aabffe460 2019-12-22T22:22:23.597 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_file_opening_threads: 16 2019-12-22T22:22:23.597 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.statistics: (nil) 2019-12-22T22:22:23.597 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.use_fsync: 0 2019-12-22T22:22:23.597 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_log_file_size: 0 2019-12-22T22:22:23.598 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_manifest_file_size: 1073741824 2019-12-22T22:22:23.598 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.log_file_time_to_roll: 0 2019-12-22T22:22:23.598 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.keep_log_file_num: 1000 2019-12-22T22:22:23.598 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.recycle_log_file_num: 0 2019-12-22T22:22:23.598 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.allow_fallocate: 1 2019-12-22T22:22:23.598 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.allow_mmap_reads: 0 2019-12-22T22:22:23.599 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.allow_mmap_writes: 0 2019-12-22T22:22:23.599 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.use_direct_reads: 0 2019-12-22T22:22:23.599 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2019-12-22T22:22:23.599 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.create_missing_column_families: 0 2019-12-22T22:22:23.599 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.db_log_dir: 2019-12-22T22:22:23.600 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2019-12-22T22:22:23.600 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.table_cache_numshardbits: 6 2019-12-22T22:22:23.600 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_subcompactions: 1 2019-12-22T22:22:23.600 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_background_flushes: -1 2019-12-22T22:22:23.600 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.WAL_ttl_seconds: 0 2019-12-22T22:22:23.600 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.WAL_size_limit_MB: 0 2019-12-22T22:22:23.601 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.manifest_preallocation_size: 4194304 2019-12-22T22:22:23.601 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.is_fd_close_on_exec: 1 2019-12-22T22:22:23.601 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.advise_random_on_open: 1 2019-12-22T22:22:23.601 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.db_write_buffer_size: 0 2019-12-22T22:22:23.601 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.write_buffer_manager: 0x555aacba69f0 2019-12-22T22:22:23.601 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.access_hint_on_compaction_start: 1 2019-12-22T22:22:23.601 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2019-12-22T22:22:23.602 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2019-12-22T22:22:23.602 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.use_adaptive_mutex: 0 2019-12-22T22:22:23.602 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.rate_limiter: (nil) 2019-12-22T22:22:23.602 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2019-12-22T22:22:23.602 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.wal_recovery_mode: 2 2019-12-22T22:22:23.602 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.enable_thread_tracking: 0 2019-12-22T22:22:23.603 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.enable_pipelined_write: 0 2019-12-22T22:22:23.603 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2019-12-22T22:22:23.603 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2019-12-22T22:22:23.603 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.write_thread_max_yield_usec: 100 2019-12-22T22:22:23.603 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2019-12-22T22:22:23.604 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.row_cache: None 2019-12-22T22:22:23.604 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.wal_filter: None 2019-12-22T22:22:23.604 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.avoid_flush_during_recovery: 0 2019-12-22T22:22:23.605 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.allow_ingest_behind: 0 2019-12-22T22:22:23.605 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.preserve_deletes: 0 2019-12-22T22:22:23.605 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.two_write_queues: 0 2019-12-22T22:22:23.605 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.manual_wal_flush: 0 2019-12-22T22:22:23.605 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.atomic_flush: 0 2019-12-22T22:22:23.605 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2019-12-22T22:22:23.606 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_background_jobs: 2 2019-12-22T22:22:23.606 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_background_compactions: -1 2019-12-22T22:22:23.606 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2019-12-22T22:22:23.606 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2019-12-22T22:22:23.606 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.delayed_write_rate : 16777216 2019-12-22T22:22:23.606 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_total_wal_size: 0 2019-12-22T22:22:23.607 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2019-12-22T22:22:23.607 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.stats_dump_period_sec: 600 2019-12-22T22:22:23.607 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.stats_persist_period_sec: 600 2019-12-22T22:22:23.607 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.stats_history_buffer_size: 1048576 2019-12-22T22:22:23.607 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_open_files: -1 2019-12-22T22:22:23.607 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.bytes_per_sync: 0 2019-12-22T22:22:23.608 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.wal_bytes_per_sync: 0 2019-12-22T22:22:23.608 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compaction_readahead_size: 0 2019-12-22T22:22:23.608 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Compression algorithms supported: 2019-12-22T22:22:23.608 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: kZSTDNotFinalCompression supported: 0 2019-12-22T22:22:23.608 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: kZSTD supported: 0 2019-12-22T22:22:23.608 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: kXpressCompression supported: 0 2019-12-22T22:22:23.608 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: kLZ4HCCompression supported: 1 2019-12-22T22:22:23.609 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: kLZ4Compression supported: 1 2019-12-22T22:22:23.609 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: kBZip2Compression supported: 0 2019-12-22T22:22:23.609 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: kZlibCompression supported: 1 2019-12-22T22:22:23.609 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: kSnappyCompression supported: 1 2019-12-22T22:22:23.609 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Fast CRC32 supported: Supported on x86 2019-12-22T22:22:23.610 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: [db/version_set.cc:3543] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000001 2019-12-22T22:22:23.610 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.610 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: [db/column_family.cc:477] --------------- Options for column family [default]: 2019-12-22T22:22:23.610 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.610 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2019-12-22T22:22:23.610 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.merge_operator: 2019-12-22T22:22:23.611 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compaction_filter: None 2019-12-22T22:22:23.611 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compaction_filter_factory: None 2019-12-22T22:22:23.611 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.memtable_factory: SkipListFactory 2019-12-22T22:22:23.611 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.table_factory: BlockBasedTable 2019-12-22T22:22:23.611 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x555aabf1a0c8) 2019-12-22T22:22:23.611 INFO:ceph.mon.c.smithi043.stdout: cache_index_and_filter_blocks: 1 2019-12-22T22:22:23.611 INFO:ceph.mon.c.smithi043.stdout: cache_index_and_filter_blocks_with_high_priority: 1 2019-12-22T22:22:23.612 INFO:ceph.mon.c.smithi043.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2019-12-22T22:22:23.612 INFO:ceph.mon.c.smithi043.stdout: pin_top_level_index_and_filter: 1 2019-12-22T22:22:23.612 INFO:ceph.mon.c.smithi043.stdout: index_type: 0 2019-12-22T22:22:23.612 INFO:ceph.mon.c.smithi043.stdout: data_block_index_type: 0 2019-12-22T22:22:23.612 INFO:ceph.mon.c.smithi043.stdout: data_block_hash_table_util_ratio: 0.750000 2019-12-22T22:22:23.612 INFO:ceph.mon.c.smithi043.stdout: hash_index_allow_collision: 1 2019-12-22T22:22:23.613 INFO:ceph.mon.c.smithi043.stdout: checksum: 1 2019-12-22T22:22:23.613 INFO:ceph.mon.c.smithi043.stdout: no_block_cache: 0 2019-12-22T22:22:23.613 INFO:ceph.mon.c.smithi043.stdout: block_cache: 0x555aabf54210 2019-12-22T22:22:23.613 INFO:ceph.mon.c.smithi043.stdout: block_cache_name: BinnedLRUCache 2019-12-22T22:22:23.613 INFO:ceph.mon.c.smithi043.stdout: block_cache_options: 2019-12-22T22:22:23.613 INFO:ceph.mon.c.smithi043.stdout: capacity : 536870912 2019-12-22T22:22:23.614 INFO:ceph.mon.c.smithi043.stdout: num_shard_bits : 4 2019-12-22T22:22:23.614 INFO:ceph.mon.c.smithi043.stdout: strict_capacity_limit : 0 2019-12-22T22:22:23.614 INFO:ceph.mon.c.smithi043.stdout: high_pri_pool_ratio: 0.000 2019-12-22T22:22:23.615 INFO:ceph.mon.c.smithi043.stdout: block_cache_compressed: (nil) 2019-12-22T22:22:23.615 INFO:ceph.mon.c.smithi043.stdout: persistent_cache: (nil) 2019-12-22T22:22:23.615 INFO:ceph.mon.c.smithi043.stdout: block_size: 4096 2019-12-22T22:22:23.615 INFO:ceph.mon.c.smithi043.stdout: block_size_deviation: 10 2019-12-22T22:22:23.615 INFO:ceph.mon.c.smithi043.stdout: block_restart_interval: 16 2019-12-22T22:22:23.615 INFO:ceph.mon.c.smithi043.stdout: index_block_restart_interval: 1 2019-12-22T22:22:23.616 INFO:ceph.mon.c.smithi043.stdout: metadata_block_size: 4096 2019-12-22T22:22:23.616 INFO:ceph.mon.c.smithi043.stdout: partition_filters: 0 2019-12-22T22:22:23.616 INFO:ceph.mon.c.smithi043.stdout: use_delta_encoding: 1 2019-12-22T22:22:23.616 INFO:ceph.mon.c.smithi043.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2019-12-22T22:22:23.616 INFO:ceph.mon.c.smithi043.stdout: whole_key_filtering: 1 2019-12-22T22:22:23.617 INFO:ceph.mon.c.smithi043.stdout: verify_compression: 0 2019-12-22T22:22:23.617 INFO:ceph.mon.c.smithi043.stdout: read_amp_bytes_per_bit: 0 2019-12-22T22:22:23.617 INFO:ceph.mon.c.smithi043.stdout: format_version: 2 2019-12-22T22:22:23.617 INFO:ceph.mon.c.smithi043.stdout: enable_index_compression: 1 2019-12-22T22:22:23.617 INFO:ceph.mon.c.smithi043.stdout: block_align: 0 2019-12-22T22:22:23.617 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.618 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.write_buffer_size: 33554432 2019-12-22T22:22:23.618 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_write_buffer_number: 2 2019-12-22T22:22:23.618 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compression: NoCompression 2019-12-22T22:22:23.618 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.bottommost_compression: Disabled 2019-12-22T22:22:23.618 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.prefix_extractor: nullptr 2019-12-22T22:22:23.619 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2019-12-22T22:22:23.619 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.num_levels: 7 2019-12-22T22:22:23.619 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2019-12-22T22:22:23.619 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2019-12-22T22:22:23.619 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2019-12-22T22:22:23.619 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2019-12-22T22:22:23.620 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2019-12-22T22:22:23.620 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2019-12-22T22:22:23.620 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2019-12-22T22:22:23.620 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2019-12-22T22:22:23.620 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compression_opts.window_bits: -14 2019-12-22T22:22:23.620 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compression_opts.level: 32767 2019-12-22T22:22:23.621 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compression_opts.strategy: 0 2019-12-22T22:22:23.621 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2019-12-22T22:22:23.621 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2019-12-22T22:22:23.621 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compression_opts.enabled: false 2019-12-22T22:22:23.621 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2019-12-22T22:22:23.622 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2019-12-22T22:22:23.622 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.level0_stop_writes_trigger: 36 2019-12-22T22:22:23.622 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.target_file_size_base: 67108864 2019-12-22T22:22:23.622 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.target_file_size_multiplier: 1 2019-12-22T22:22:23.622 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2019-12-22T22:22:23.622 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2019-12-22T22:22:23.623 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2019-12-22T22:22:23.623 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2019-12-22T22:22:23.623 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2019-12-22T22:22:23.623 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2019-12-22T22:22:23.623 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2019-12-22T22:22:23.623 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2019-12-22T22:22:23.623 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2019-12-22T22:22:23.624 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2019-12-22T22:22:23.624 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2019-12-22T22:22:23.624 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_compaction_bytes: 1677721600 2019-12-22T22:22:23.624 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.arena_block_size: 4194304 2019-12-22T22:22:23.624 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2019-12-22T22:22:23.625 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2019-12-22T22:22:23.625 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2019-12-22T22:22:23.625 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.disable_auto_compactions: 0 2019-12-22T22:22:23.626 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2019-12-22T22:22:23.626 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2019-12-22T22:22:23.626 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2019-12-22T22:22:23.626 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2019-12-22T22:22:23.626 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2019-12-22T22:22:23.626 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2019-12-22T22:22:23.627 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2019-12-22T22:22:23.627 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2019-12-22T22:22:23.627 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2019-12-22T22:22:23.627 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2019-12-22T22:22:23.627 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.table_properties_collectors: 2019-12-22T22:22:23.628 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.inplace_update_support: 0 2019-12-22T22:22:23.628 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.inplace_update_num_locks: 10000 2019-12-22T22:22:23.628 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2019-12-22T22:22:23.628 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.memtable_whole_key_filtering: 0 2019-12-22T22:22:23.628 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.memtable_huge_page_size: 0 2019-12-22T22:22:23.629 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.bloom_locality: 0 2019-12-22T22:22:23.629 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.max_successive_merges: 0 2019-12-22T22:22:23.629 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.optimize_filters_for_hits: 0 2019-12-22T22:22:23.629 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.paranoid_file_checks: 0 2019-12-22T22:22:23.629 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.force_consistency_checks: 0 2019-12-22T22:22:23.629 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.report_bg_io_stats: 0 2019-12-22T22:22:23.630 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.706+0000 7f5c12ae36c0 4 rocksdb: Options.ttl: 0 2019-12-22T22:22:23.630 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.707+0000 7f5c12ae36c0 4 rocksdb: [db/version_set.cc:3757] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2019-12-22T22:22:23.630 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.630 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.707+0000 7f5c12ae36c0 4 rocksdb: [db/version_set.cc:3766] Column family [default] (ID 0), log number is 0 2019-12-22T22:22:23.630 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.630 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.707+0000 7f5c12ae36c0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1577053342709155, "job": 1, "event": "recovery_started", "log_files": [3]} 2019-12-22T22:22:23.630 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.707+0000 7f5c12ae36c0 4 rocksdb: [db/db_impl_open.cc:583] Recovering log #3 mode 2 2019-12-22T22:22:23.631 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.708+0000 7f5c12ae36c0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1577053342709896, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 4, "file_size": 1368, "table_properties": {"data_size": 466, "index_size": 28, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 344, "raw_average_value_size": 68, "num_data_blocks": 1, "num_entries": 5, "filter_policy_name": "rocksdb.BuiltinBloomFilter"}} 2019-12-22T22:22:23.631 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.708+0000 7f5c12ae36c0 4 rocksdb: [db/version_set.cc:3036] Creating manifest 5 2019-12-22T22:22:23.631 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.631 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.708+0000 7f5c12ae36c0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1577053342710392, "job": 1, "event": "recovery_finished"} 2019-12-22T22:22:23.631 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.710+0000 7f5c12ae36c0 4 rocksdb: DB pointer 0x555aacb89200 2019-12-22T22:22:23.631 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.710+0000 7f5bf9c16700 4 rocksdb: [db/db_impl.cc:777] ------- DUMPING STATS ------- 2019-12-22T22:22:23.632 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.710+0000 7f5bf9c16700 4 rocksdb: [db/db_impl.cc:778] 2019-12-22T22:22:23.632 INFO:ceph.mon.c.smithi043.stdout:** DB Stats ** 2019-12-22T22:22:23.632 INFO:ceph.mon.c.smithi043.stdout:Uptime(secs): 0.0 total, 0.0 interval 2019-12-22T22:22:23.632 INFO:ceph.mon.c.smithi043.stdout:Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2019-12-22T22:22:23.632 INFO:ceph.mon.c.smithi043.stdout:Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2019-12-22T22:22:23.632 INFO:ceph.mon.c.smithi043.stdout:Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2019-12-22T22:22:23.633 INFO:ceph.mon.c.smithi043.stdout:Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2019-12-22T22:22:23.633 INFO:ceph.mon.c.smithi043.stdout:Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2019-12-22T22:22:23.633 INFO:ceph.mon.c.smithi043.stdout:Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2019-12-22T22:22:23.633 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.633 INFO:ceph.mon.c.smithi043.stdout:** Compaction Stats [default] ** 2019-12-22T22:22:23.633 INFO:ceph.mon.c.smithi043.stdout:Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2019-12-22T22:22:23.634 INFO:ceph.mon.c.smithi043.stdout:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2019-12-22T22:22:23.634 INFO:ceph.mon.c.smithi043.stdout: L0 1/0 1.34 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 2019-12-22T22:22:23.634 INFO:ceph.mon.c.smithi043.stdout: Sum 1/0 1.34 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 2019-12-22T22:22:23.634 INFO:ceph.mon.c.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 2019-12-22T22:22:23.634 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.634 INFO:ceph.mon.c.smithi043.stdout:** Compaction Stats [default] ** 2019-12-22T22:22:23.635 INFO:ceph.mon.c.smithi043.stdout:Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2019-12-22T22:22:23.635 INFO:ceph.mon.c.smithi043.stdout:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2019-12-22T22:22:23.636 INFO:ceph.mon.c.smithi043.stdout:User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.0 0.00 0.00 1 0.001 0 0 2019-12-22T22:22:23.636 INFO:ceph.mon.c.smithi043.stdout:Uptime(secs): 0.0 total, 0.0 interval 2019-12-22T22:22:23.636 INFO:ceph.mon.c.smithi043.stdout:Flush(GB): cumulative 0.000, interval 0.000 2019-12-22T22:22:23.636 INFO:ceph.mon.c.smithi043.stdout:AddFile(GB): cumulative 0.000, interval 0.000 2019-12-22T22:22:23.636 INFO:ceph.mon.c.smithi043.stdout:AddFile(Total Files): cumulative 0, interval 0 2019-12-22T22:22:23.636 INFO:ceph.mon.c.smithi043.stdout:AddFile(L0 Files): cumulative 0, interval 0 2019-12-22T22:22:23.637 INFO:ceph.mon.c.smithi043.stdout:AddFile(Keys): cumulative 0, interval 0 2019-12-22T22:22:23.637 INFO:ceph.mon.c.smithi043.stdout:Cumulative compaction: 0.00 GB write, 0.39 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2019-12-22T22:22:23.637 INFO:ceph.mon.c.smithi043.stdout:Interval compaction: 0.00 GB write, 0.39 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2019-12-22T22:22:23.637 INFO:ceph.mon.c.smithi043.stdout:Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2019-12-22T22:22:23.637 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.637 INFO:ceph.mon.c.smithi043.stdout:** File Read Latency Histogram By Level [default] ** 2019-12-22T22:22:23.637 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.638 INFO:ceph.mon.c.smithi043.stdout:** Compaction Stats [default] ** 2019-12-22T22:22:23.638 INFO:ceph.mon.c.smithi043.stdout:Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2019-12-22T22:22:23.638 INFO:ceph.mon.c.smithi043.stdout:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2019-12-22T22:22:23.638 INFO:ceph.mon.c.smithi043.stdout: L0 1/0 1.34 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 2019-12-22T22:22:23.638 INFO:ceph.mon.c.smithi043.stdout: Sum 1/0 1.34 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.0 0.00 0.00 1 0.001 0 0 2019-12-22T22:22:23.638 INFO:ceph.mon.c.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2019-12-22T22:22:23.638 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.639 INFO:ceph.mon.c.smithi043.stdout:** Compaction Stats [default] ** 2019-12-22T22:22:23.639 INFO:ceph.mon.c.smithi043.stdout:Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2019-12-22T22:22:23.639 INFO:ceph.mon.c.smithi043.stdout:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2019-12-22T22:22:23.639 INFO:ceph.mon.c.smithi043.stdout:User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.0 0.00 0.00 1 0.001 0 0 2019-12-22T22:22:23.639 INFO:ceph.mon.c.smithi043.stdout:Uptime(secs): 0.0 total, 0.0 interval 2019-12-22T22:22:23.639 INFO:ceph.mon.c.smithi043.stdout:Flush(GB): cumulative 0.000, interval 0.000 2019-12-22T22:22:23.640 INFO:ceph.mon.c.smithi043.stdout:AddFile(GB): cumulative 0.000, interval 0.000 2019-12-22T22:22:23.640 INFO:ceph.mon.c.smithi043.stdout:AddFile(Total Files): cumulative 0, interval 0 2019-12-22T22:22:23.640 INFO:ceph.mon.c.smithi043.stdout:AddFile(L0 Files): cumulative 0, interval 0 2019-12-22T22:22:23.640 INFO:ceph.mon.c.smithi043.stdout:AddFile(Keys): cumulative 0, interval 0 2019-12-22T22:22:23.640 INFO:ceph.mon.c.smithi043.stdout:Cumulative compaction: 0.00 GB write, 0.39 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2019-12-22T22:22:23.640 INFO:ceph.mon.c.smithi043.stdout:Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2019-12-22T22:22:23.640 INFO:ceph.mon.c.smithi043.stdout:Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2019-12-22T22:22:23.641 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.641 INFO:ceph.mon.c.smithi043.stdout:** File Read Latency Histogram By Level [default] ** 2019-12-22T22:22:23.641 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.641 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.710+0000 7f5c12ae36c0 0 mon.c does not exist in monmap, will attempt to join an existing cluster 2019-12-22T22:22:23.641 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.710+0000 7f5c12ae36c0 0 using public_addrv v1:172.21.15.43:6789/0 2019-12-22T22:22:23.641 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.711+0000 7f5c12ae36c0 0 starting mon.c rank -1 at public addrs v1:172.21.15.43:6789/0 at bind addrs v1:172.21.15.43:6789/0 mon_data /var/lib/ceph/mon/ceph-c fsid 147630da-2509-11ea-8284-001a4aab830c 2019-12-22T22:22:23.641 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.712+0000 7f5c12ae36c0 1 mon.c@-1(???) e0 preinit fsid 147630da-2509-11ea-8284-001a4aab830c 2019-12-22T22:22:23.642 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.734+0000 7f5bf7c12700 4 rocksdb: [db/db_impl_write.cc:1470] [default] New memtable created with log file: #9. Immutable memtables: 0. 2019-12-22T22:22:23.642 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.642 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.734+0000 7f5c01c26700 4 rocksdb: (Original Log Time 2019/12/22-22:22:22.736211) [db/db_impl_compaction_flush.cc:2073] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 2019-12-22T22:22:23.642 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.734+0000 7f5c01c26700 4 rocksdb: [db/flush_job.cc:322] [default] [JOB 3] Flushing memtable with next log file: 9 2019-12-22T22:22:23.642 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.642 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.734+0000 7f5c01c26700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1577053342736284, "job": 3, "event": "flush_started", "num_memtables": 1, "num_entries": 845, "num_deletes": 31, "total_data_size": 2114753, "memory_usage": 2130472, "flush_reason": "Manual Compaction"} 2019-12-22T22:22:23.642 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.734+0000 7f5c01c26700 4 rocksdb: [db/flush_job.cc:351] [default] [JOB 3] Level-0 flush table #10: started 2019-12-22T22:22:23.643 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.736+0000 7f5bfc41b700 0 mon.c@-1(synchronizing).mds e1 new map 2019-12-22T22:22:23.643 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.736+0000 7f5bfc41b700 0 mon.c@-1(synchronizing).mds e1 print_map 2019-12-22T22:22:23.643 INFO:ceph.mon.c.smithi043.stdout:e1 2019-12-22T22:22:23.643 INFO:ceph.mon.c.smithi043.stdout:enable_multiple, ever_enabled_multiple: 0,0 2019-12-22T22:22:23.643 INFO:ceph.mon.c.smithi043.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2019-12-22T22:22:23.643 INFO:ceph.mon.c.smithi043.stdout:legacy client fscid: -1 2019-12-22T22:22:23.643 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.644 INFO:ceph.mon.c.smithi043.stdout:No filesystems configured 2019-12-22T22:22:23.644 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.644 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.736+0000 7f5bfc41b700 1 mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2019-12-22T22:22:23.644 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.736+0000 7f5bfc41b700 1 mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2019-12-22T22:22:23.644 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.736+0000 7f5bfc41b700 1 mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2019-12-22T22:22:23.644 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.736+0000 7f5bfc41b700 0 mon.c@-1(synchronizing).osd e3 crush map has features 288514050185494528, adjusting msgr requires 2019-12-22T22:22:23.644 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.736+0000 7f5bfc41b700 0 mon.c@-1(synchronizing).osd e3 crush map has features 288514050185494528, adjusting msgr requires 2019-12-22T22:22:23.645 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.736+0000 7f5bfc41b700 0 mon.c@-1(synchronizing).osd e3 crush map has features 3314932999778484224, adjusting msgr requires 2019-12-22T22:22:23.645 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.736+0000 7f5bfc41b700 0 mon.c@-1(synchronizing).osd e3 crush map has features 288514050185494528, adjusting msgr requires 2019-12-22T22:22:23.645 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.736+0000 7f5bfc41b700 1 mon.c@-1(synchronizing).paxosservice(auth 1..5) refresh upgraded, format 0 -> 3 2019-12-22T22:22:23.645 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.738+0000 7f5c01c26700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1577053342739774, "cf_name": "default", "job": 3, "event": "table_file_creation", "file_number": 10, "file_size": 1961440, "table_properties": {"data_size": 1956050, "index_size": 3351, "filter_size": 1221, "raw_key_size": 16062, "raw_average_key_size": 33, "raw_value_size": 1944198, "raw_average_value_size": 4033, "num_data_blocks": 111, "num_entries": 482, "filter_policy_name": "rocksdb.BuiltinBloomFilter"}} 2019-12-22T22:22:23.645 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.738+0000 7f5c01c26700 4 rocksdb: [db/flush_job.cc:392] [default] [JOB 3] Level-0 flush table #10: 1961440 bytes OK 2019-12-22T22:22:23.645 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.738+0000 7f5c01c26700 4 rocksdb: (Original Log Time 2019/12/22-22:22:22.739808) [db/memtable_list.cc:380] [default] Level-0 commit table #10 started 2019-12-22T22:22:23.646 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.738+0000 7f5c01c26700 4 rocksdb: (Original Log Time 2019/12/22-22:22:22.739941) [db/memtable_list.cc:428] [default] Level-0 commit table #10: memtable #1 done 2019-12-22T22:22:23.646 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.738+0000 7f5c01c26700 4 rocksdb: (Original Log Time 2019/12/22-22:22:22.739954) EVENT_LOG_v1 {"time_micros": 1577053342739947, "job": 3, "event": "flush_finished", "output_compression": "NoCompression", "lsm_state": [2, 0, 0, 0, 0, 0, 0], "immutable_memtables": 0} 2019-12-22T22:22:23.646 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.738+0000 7f5c01c26700 4 rocksdb: (Original Log Time 2019/12/22-22:22:22.739964) [db/db_impl_compaction_flush.cc:201] [default] Level summary: files[2 0 0 0 0 0 0] max score 0.50 2019-12-22T22:22:23.646 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.646 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.738+0000 7f5c01c26700 4 rocksdb: [db/db_impl_files.cc:356] [JOB 3] Try to delete WAL files size 2108831, prev total WAL file size 2353434, number of live WAL files 2. 2019-12-22T22:22:23.647 INFO:ceph.mon.c.smithi043.stdout: 2019-12-22T22:22:23.647 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.738+0000 7f5bf7c12700 4 rocksdb: [db/db_impl_compaction_flush.cc:1403] [default] Manual compaction starting 2019-12-22T22:22:23.647 INFO:ceph.mon.c.smithi043.stdout:2019-12-22T22:22:22.738+0000 7f5bfc41b700 10 mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete,mon_osdmap_full_prune_interval,mon_osdmap_full_prune_min,mon_osdmap_full_prune_txsize 2019-12-22T22:22:28.028 INFO:teuthology.orchestra.run.smithi043.stdout: 2019-12-22T22:22:28.029 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":2,"fsid":"147630da-2509-11ea-8284-001a4aab830c","modified":"2019-12-22T22:22:22.746062Z","created":"2019-12-22T22:19:57.173839Z","min_mon_release":15,"min_mon_release_name":"octopus","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.91:6789","nonce":0}]},"addr":"172.21.15.91:6789/0","public_addr":"172.21.15.91:6789/0","priority":0,"weight":0},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0}],"quorum":[0,1]} 2019-12-22T22:22:28.029 INFO:teuthology.orchestra.run.smithi043.stderr:dumped monmap epoch 2 2019-12-22T22:22:28.535 INFO:tasks.cephadm:Adding mon.b on smithi107 2019-12-22T22:22:28.536 INFO:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph orchestrator mon update 3 'smithi107:[v1:172.21.15.107:6789]=b' 2019-12-22T22:22:31.079 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi043,172.21.15.43' (ECDSA) to the list of known hosts. 2019-12-22T22:22:32.080 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi107,172.21.15.107' (ECDSA) to the list of known hosts. 2019-12-22T22:22:35.992 INFO:teuthology.orchestra.run.smithi107.stdout:["(Re)deployed mon.b on host 'smithi107'"] 2019-12-22T22:22:36.653 INFO:teuthology.orchestra.run.smithi107:mon.b> sudo /home/ubuntu/cephtest/cephadm logs -f --fsid 147630da-2509-11ea-8284-001a4aab830c --name mon.b 2019-12-22T22:22:36.659 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2019-12-22T22:22:36.659 INFO:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph mon dump -f json 2019-12-22T22:22:37.057 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.506+0000 7f7da21566c0 0 set uid:gid to 167:167 (ceph:ceph) 2019-12-22T22:22:37.057 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.506+0000 7f7da21566c0 0 ceph version 15.0.0-8807-g99b509f (99b509ffd4dd354919b08b38ba83b998fbfdc107) octopus (dev), process ceph-mon, pid 1 2019-12-22T22:22:37.057 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.506+0000 7f7da21566c0 0 pidfile_write: ignore empty --pid-file 2019-12-22T22:22:37.057 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.516+0000 7f7da21566c0 0 load: jerasure load: lrc load: isa 2019-12-22T22:22:37.057 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.516+0000 7f7da21566c0 0 set rocksdb option compression = kNoCompression 2019-12-22T22:22:37.058 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.516+0000 7f7da21566c0 0 set rocksdb option level_compaction_dynamic_level_bytes = true 2019-12-22T22:22:37.058 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.516+0000 7f7da21566c0 0 set rocksdb option write_buffer_size = 33554432 2019-12-22T22:22:37.058 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.516+0000 7f7da21566c0 0 set rocksdb option compression = kNoCompression 2019-12-22T22:22:37.058 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.516+0000 7f7da21566c0 0 set rocksdb option level_compaction_dynamic_level_bytes = true 2019-12-22T22:22:37.059 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.516+0000 7f7da21566c0 0 set rocksdb option write_buffer_size = 33554432 2019-12-22T22:22:37.059 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 1 rocksdb: do_open column families: [default] 2019-12-22T22:22:37.059 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: RocksDB version: 6.1.2 2019-12-22T22:22:37.059 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.059 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2019-12-22T22:22:37.060 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Compile date Dec 22 2019 2019-12-22T22:22:37.060 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: DB SUMMARY 2019-12-22T22:22:37.060 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.060 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: CURRENT file: CURRENT 2019-12-22T22:22:37.060 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.060 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: IDENTITY file: IDENTITY 2019-12-22T22:22:37.060 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.061 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: MANIFEST file: MANIFEST-000001 size: 13 Bytes 2019-12-22T22:22:37.061 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.061 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2019-12-22T22:22:37.061 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.086 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000003.log size: 540 ; 2019-12-22T22:22:37.086 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.086 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.error_if_exists: 0 2019-12-22T22:22:37.086 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.create_if_missing: 0 2019-12-22T22:22:37.087 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.paranoid_checks: 1 2019-12-22T22:22:37.087 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.env: 0x562ebcb6c260 2019-12-22T22:22:37.087 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.info_log: 0x562ebe4e64a0 2019-12-22T22:22:37.087 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.max_file_opening_threads: 16 2019-12-22T22:22:37.088 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.statistics: (nil) 2019-12-22T22:22:37.088 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.use_fsync: 0 2019-12-22T22:22:37.088 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.max_log_file_size: 0 2019-12-22T22:22:37.088 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.max_manifest_file_size: 1073741824 2019-12-22T22:22:37.089 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.log_file_time_to_roll: 0 2019-12-22T22:22:37.089 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.keep_log_file_num: 1000 2019-12-22T22:22:37.089 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.recycle_log_file_num: 0 2019-12-22T22:22:37.089 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.allow_fallocate: 1 2019-12-22T22:22:37.090 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.allow_mmap_reads: 0 2019-12-22T22:22:37.090 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.allow_mmap_writes: 0 2019-12-22T22:22:37.090 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.use_direct_reads: 0 2019-12-22T22:22:37.090 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2019-12-22T22:22:37.091 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.create_missing_column_families: 0 2019-12-22T22:22:37.091 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.db_log_dir: 2019-12-22T22:22:37.091 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2019-12-22T22:22:37.091 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.table_cache_numshardbits: 6 2019-12-22T22:22:37.091 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.max_subcompactions: 1 2019-12-22T22:22:37.092 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.max_background_flushes: -1 2019-12-22T22:22:37.092 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.WAL_ttl_seconds: 0 2019-12-22T22:22:37.092 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.WAL_size_limit_MB: 0 2019-12-22T22:22:37.093 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.manifest_preallocation_size: 4194304 2019-12-22T22:22:37.094 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.is_fd_close_on_exec: 1 2019-12-22T22:22:37.094 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.advise_random_on_open: 1 2019-12-22T22:22:37.094 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.db_write_buffer_size: 0 2019-12-22T22:22:37.096 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.write_buffer_manager: 0x562ebf08ea50 2019-12-22T22:22:37.096 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.access_hint_on_compaction_start: 1 2019-12-22T22:22:37.096 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2019-12-22T22:22:37.096 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2019-12-22T22:22:37.097 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.use_adaptive_mutex: 0 2019-12-22T22:22:37.097 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.rate_limiter: (nil) 2019-12-22T22:22:37.097 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2019-12-22T22:22:37.097 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.wal_recovery_mode: 2 2019-12-22T22:22:37.098 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.enable_thread_tracking: 0 2019-12-22T22:22:37.098 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.enable_pipelined_write: 0 2019-12-22T22:22:37.098 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2019-12-22T22:22:37.098 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2019-12-22T22:22:37.099 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.write_thread_max_yield_usec: 100 2019-12-22T22:22:37.099 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2019-12-22T22:22:37.099 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.row_cache: None 2019-12-22T22:22:37.099 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.wal_filter: None 2019-12-22T22:22:37.100 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.avoid_flush_during_recovery: 0 2019-12-22T22:22:37.100 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.allow_ingest_behind: 0 2019-12-22T22:22:37.100 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.preserve_deletes: 0 2019-12-22T22:22:37.100 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.two_write_queues: 0 2019-12-22T22:22:37.100 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.manual_wal_flush: 0 2019-12-22T22:22:37.101 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.atomic_flush: 0 2019-12-22T22:22:37.101 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2019-12-22T22:22:37.101 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.max_background_jobs: 2 2019-12-22T22:22:37.101 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.max_background_compactions: -1 2019-12-22T22:22:37.102 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2019-12-22T22:22:37.102 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2019-12-22T22:22:37.102 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.delayed_write_rate : 16777216 2019-12-22T22:22:37.102 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.max_total_wal_size: 0 2019-12-22T22:22:37.102 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2019-12-22T22:22:37.103 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.stats_dump_period_sec: 600 2019-12-22T22:22:37.103 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.stats_persist_period_sec: 600 2019-12-22T22:22:37.103 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.stats_history_buffer_size: 1048576 2019-12-22T22:22:37.103 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.max_open_files: -1 2019-12-22T22:22:37.104 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.bytes_per_sync: 0 2019-12-22T22:22:37.104 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.wal_bytes_per_sync: 0 2019-12-22T22:22:37.104 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Options.compaction_readahead_size: 0 2019-12-22T22:22:37.104 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Compression algorithms supported: 2019-12-22T22:22:37.104 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: kZSTDNotFinalCompression supported: 0 2019-12-22T22:22:37.105 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: kZSTD supported: 0 2019-12-22T22:22:37.105 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: kXpressCompression supported: 0 2019-12-22T22:22:37.106 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: kLZ4HCCompression supported: 1 2019-12-22T22:22:37.106 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: kLZ4Compression supported: 1 2019-12-22T22:22:37.106 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: kBZip2Compression supported: 0 2019-12-22T22:22:37.106 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: kZlibCompression supported: 1 2019-12-22T22:22:37.106 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: kSnappyCompression supported: 1 2019-12-22T22:22:37.107 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.517+0000 7f7da21566c0 4 rocksdb: Fast CRC32 supported: Supported on x86 2019-12-22T22:22:37.107 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: [db/version_set.cc:3543] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000001 2019-12-22T22:22:37.107 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.107 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: [db/column_family.cc:477] --------------- Options for column family [default]: 2019-12-22T22:22:37.108 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.108 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2019-12-22T22:22:37.108 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.merge_operator: 2019-12-22T22:22:37.108 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.compaction_filter: None 2019-12-22T22:22:37.108 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.compaction_filter_factory: None 2019-12-22T22:22:37.109 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.memtable_factory: SkipListFactory 2019-12-22T22:22:37.109 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.table_factory: BlockBasedTable 2019-12-22T22:22:37.109 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562ebe4020c8) 2019-12-22T22:22:37.109 INFO:ceph.mon.b.smithi107.stdout: cache_index_and_filter_blocks: 1 2019-12-22T22:22:37.109 INFO:ceph.mon.b.smithi107.stdout: cache_index_and_filter_blocks_with_high_priority: 1 2019-12-22T22:22:37.110 INFO:ceph.mon.b.smithi107.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2019-12-22T22:22:37.110 INFO:ceph.mon.b.smithi107.stdout: pin_top_level_index_and_filter: 1 2019-12-22T22:22:37.110 INFO:ceph.mon.b.smithi107.stdout: index_type: 0 2019-12-22T22:22:37.110 INFO:ceph.mon.b.smithi107.stdout: data_block_index_type: 0 2019-12-22T22:22:37.111 INFO:ceph.mon.b.smithi107.stdout: data_block_hash_table_util_ratio: 0.750000 2019-12-22T22:22:37.111 INFO:ceph.mon.b.smithi107.stdout: hash_index_allow_collision: 1 2019-12-22T22:22:37.111 INFO:ceph.mon.b.smithi107.stdout: checksum: 1 2019-12-22T22:22:37.111 INFO:ceph.mon.b.smithi107.stdout: no_block_cache: 0 2019-12-22T22:22:37.111 INFO:ceph.mon.b.smithi107.stdout: block_cache: 0x562ebe43c210 2019-12-22T22:22:37.112 INFO:ceph.mon.b.smithi107.stdout: block_cache_name: BinnedLRUCache 2019-12-22T22:22:37.112 INFO:ceph.mon.b.smithi107.stdout: block_cache_options: 2019-12-22T22:22:37.112 INFO:ceph.mon.b.smithi107.stdout: capacity : 536870912 2019-12-22T22:22:37.112 INFO:ceph.mon.b.smithi107.stdout: num_shard_bits : 4 2019-12-22T22:22:37.112 INFO:ceph.mon.b.smithi107.stdout: strict_capacity_limit : 0 2019-12-22T22:22:37.113 INFO:ceph.mon.b.smithi107.stdout: high_pri_pool_ratio: 0.000 2019-12-22T22:22:37.113 INFO:ceph.mon.b.smithi107.stdout: block_cache_compressed: (nil) 2019-12-22T22:22:37.113 INFO:ceph.mon.b.smithi107.stdout: persistent_cache: (nil) 2019-12-22T22:22:37.113 INFO:ceph.mon.b.smithi107.stdout: block_size: 4096 2019-12-22T22:22:37.113 INFO:ceph.mon.b.smithi107.stdout: block_size_deviation: 10 2019-12-22T22:22:37.114 INFO:ceph.mon.b.smithi107.stdout: block_restart_interval: 16 2019-12-22T22:22:37.114 INFO:ceph.mon.b.smithi107.stdout: index_block_restart_interval: 1 2019-12-22T22:22:37.114 INFO:ceph.mon.b.smithi107.stdout: metadata_block_size: 4096 2019-12-22T22:22:37.114 INFO:ceph.mon.b.smithi107.stdout: partition_filters: 0 2019-12-22T22:22:37.115 INFO:ceph.mon.b.smithi107.stdout: use_delta_encoding: 1 2019-12-22T22:22:37.115 INFO:ceph.mon.b.smithi107.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2019-12-22T22:22:37.115 INFO:ceph.mon.b.smithi107.stdout: whole_key_filtering: 1 2019-12-22T22:22:37.116 INFO:ceph.mon.b.smithi107.stdout: verify_compression: 0 2019-12-22T22:22:37.116 INFO:ceph.mon.b.smithi107.stdout: read_amp_bytes_per_bit: 0 2019-12-22T22:22:37.116 INFO:ceph.mon.b.smithi107.stdout: format_version: 2 2019-12-22T22:22:37.117 INFO:ceph.mon.b.smithi107.stdout: enable_index_compression: 1 2019-12-22T22:22:37.117 INFO:ceph.mon.b.smithi107.stdout: block_align: 0 2019-12-22T22:22:37.117 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.117 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.write_buffer_size: 33554432 2019-12-22T22:22:37.118 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.max_write_buffer_number: 2 2019-12-22T22:22:37.118 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.compression: NoCompression 2019-12-22T22:22:37.118 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.bottommost_compression: Disabled 2019-12-22T22:22:37.118 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.prefix_extractor: nullptr 2019-12-22T22:22:37.118 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2019-12-22T22:22:37.119 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.num_levels: 7 2019-12-22T22:22:37.119 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2019-12-22T22:22:37.119 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2019-12-22T22:22:37.119 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2019-12-22T22:22:37.119 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2019-12-22T22:22:37.119 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2019-12-22T22:22:37.119 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2019-12-22T22:22:37.120 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2019-12-22T22:22:37.120 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2019-12-22T22:22:37.120 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.compression_opts.window_bits: -14 2019-12-22T22:22:37.120 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.compression_opts.level: 32767 2019-12-22T22:22:37.120 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.compression_opts.strategy: 0 2019-12-22T22:22:37.120 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2019-12-22T22:22:37.121 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2019-12-22T22:22:37.121 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.compression_opts.enabled: false 2019-12-22T22:22:37.121 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2019-12-22T22:22:37.122 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2019-12-22T22:22:37.123 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.level0_stop_writes_trigger: 36 2019-12-22T22:22:37.123 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.target_file_size_base: 67108864 2019-12-22T22:22:37.123 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.target_file_size_multiplier: 1 2019-12-22T22:22:37.123 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2019-12-22T22:22:37.123 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2019-12-22T22:22:37.124 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2019-12-22T22:22:37.124 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2019-12-22T22:22:37.124 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2019-12-22T22:22:37.124 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2019-12-22T22:22:37.124 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2019-12-22T22:22:37.124 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2019-12-22T22:22:37.125 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2019-12-22T22:22:37.125 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2019-12-22T22:22:37.125 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2019-12-22T22:22:37.125 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.max_compaction_bytes: 1677721600 2019-12-22T22:22:37.125 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.arena_block_size: 4194304 2019-12-22T22:22:37.125 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2019-12-22T22:22:37.125 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2019-12-22T22:22:37.126 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2019-12-22T22:22:37.126 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.disable_auto_compactions: 0 2019-12-22T22:22:37.126 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2019-12-22T22:22:37.126 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2019-12-22T22:22:37.126 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2019-12-22T22:22:37.126 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2019-12-22T22:22:37.126 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2019-12-22T22:22:37.127 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2019-12-22T22:22:37.127 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2019-12-22T22:22:37.127 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2019-12-22T22:22:37.127 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2019-12-22T22:22:37.127 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2019-12-22T22:22:37.127 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.table_properties_collectors: 2019-12-22T22:22:37.128 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.inplace_update_support: 0 2019-12-22T22:22:37.128 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.inplace_update_num_locks: 10000 2019-12-22T22:22:37.128 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2019-12-22T22:22:37.128 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.memtable_whole_key_filtering: 0 2019-12-22T22:22:37.128 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.memtable_huge_page_size: 0 2019-12-22T22:22:37.128 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.bloom_locality: 0 2019-12-22T22:22:37.128 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.max_successive_merges: 0 2019-12-22T22:22:37.129 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.optimize_filters_for_hits: 0 2019-12-22T22:22:37.129 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.paranoid_file_checks: 0 2019-12-22T22:22:37.129 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.force_consistency_checks: 0 2019-12-22T22:22:37.129 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.report_bg_io_stats: 0 2019-12-22T22:22:37.129 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.518+0000 7f7da21566c0 4 rocksdb: Options.ttl: 0 2019-12-22T22:22:37.129 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.519+0000 7f7da21566c0 4 rocksdb: [db/version_set.cc:3757] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2019-12-22T22:22:37.130 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.130 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.519+0000 7f7da21566c0 4 rocksdb: [db/version_set.cc:3766] Column family [default] (ID 0), log number is 0 2019-12-22T22:22:37.130 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.130 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.519+0000 7f7da21566c0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1577053356520594, "job": 1, "event": "recovery_started", "log_files": [3]} 2019-12-22T22:22:37.130 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.519+0000 7f7da21566c0 4 rocksdb: [db/db_impl_open.cc:583] Recovering log #3 mode 2 2019-12-22T22:22:37.130 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.520+0000 7f7da21566c0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1577053356521785, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 4, "file_size": 1454, "table_properties": {"data_size": 552, "index_size": 28, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 430, "raw_average_value_size": 86, "num_data_blocks": 1, "num_entries": 5, "filter_policy_name": "rocksdb.BuiltinBloomFilter"}} 2019-12-22T22:22:37.130 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.520+0000 7f7da21566c0 4 rocksdb: [db/version_set.cc:3036] Creating manifest 5 2019-12-22T22:22:37.131 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.131 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.521+0000 7f7da21566c0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1577053356522635, "job": 1, "event": "recovery_finished"} 2019-12-22T22:22:37.131 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.523+0000 7f7da21566c0 4 rocksdb: DB pointer 0x562ebf071200 2019-12-22T22:22:37.131 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.523+0000 7f7d89289700 4 rocksdb: [db/db_impl.cc:777] ------- DUMPING STATS ------- 2019-12-22T22:22:37.131 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.523+0000 7f7d89289700 4 rocksdb: [db/db_impl.cc:778] 2019-12-22T22:22:37.131 INFO:ceph.mon.b.smithi107.stdout:** DB Stats ** 2019-12-22T22:22:37.131 INFO:ceph.mon.b.smithi107.stdout:Uptime(secs): 0.0 total, 0.0 interval 2019-12-22T22:22:37.131 INFO:ceph.mon.b.smithi107.stdout:Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2019-12-22T22:22:37.132 INFO:ceph.mon.b.smithi107.stdout:Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2019-12-22T22:22:37.132 INFO:ceph.mon.b.smithi107.stdout:Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2019-12-22T22:22:37.132 INFO:ceph.mon.b.smithi107.stdout:Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2019-12-22T22:22:37.132 INFO:ceph.mon.b.smithi107.stdout:Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2019-12-22T22:22:37.132 INFO:ceph.mon.b.smithi107.stdout:Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2019-12-22T22:22:37.132 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.133 INFO:ceph.mon.b.smithi107.stdout:** Compaction Stats [default] ** 2019-12-22T22:22:37.133 INFO:ceph.mon.b.smithi107.stdout:Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2019-12-22T22:22:37.133 INFO:ceph.mon.b.smithi107.stdout:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2019-12-22T22:22:37.133 INFO:ceph.mon.b.smithi107.stdout: L0 1/0 1.42 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.4 0.00 0.00 1 0.001 0 0 2019-12-22T22:22:37.133 INFO:ceph.mon.b.smithi107.stdout: Sum 1/0 1.42 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.4 0.00 0.00 1 0.001 0 0 2019-12-22T22:22:37.133 INFO:ceph.mon.b.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.4 0.00 0.00 1 0.001 0 0 2019-12-22T22:22:37.133 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.134 INFO:ceph.mon.b.smithi107.stdout:** Compaction Stats [default] ** 2019-12-22T22:22:37.134 INFO:ceph.mon.b.smithi107.stdout:Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2019-12-22T22:22:37.134 INFO:ceph.mon.b.smithi107.stdout:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2019-12-22T22:22:37.134 INFO:ceph.mon.b.smithi107.stdout:User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.4 0.00 0.00 1 0.001 0 0 2019-12-22T22:22:37.134 INFO:ceph.mon.b.smithi107.stdout:Uptime(secs): 0.0 total, 0.0 interval 2019-12-22T22:22:37.135 INFO:ceph.mon.b.smithi107.stdout:Flush(GB): cumulative 0.000, interval 0.000 2019-12-22T22:22:37.135 INFO:ceph.mon.b.smithi107.stdout:AddFile(GB): cumulative 0.000, interval 0.000 2019-12-22T22:22:37.135 INFO:ceph.mon.b.smithi107.stdout:AddFile(Total Files): cumulative 0, interval 0 2019-12-22T22:22:37.136 INFO:ceph.mon.b.smithi107.stdout:AddFile(L0 Files): cumulative 0, interval 0 2019-12-22T22:22:37.136 INFO:ceph.mon.b.smithi107.stdout:AddFile(Keys): cumulative 0, interval 0 2019-12-22T22:22:37.136 INFO:ceph.mon.b.smithi107.stdout:Cumulative compaction: 0.00 GB write, 0.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2019-12-22T22:22:37.136 INFO:ceph.mon.b.smithi107.stdout:Interval compaction: 0.00 GB write, 0.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2019-12-22T22:22:37.138 INFO:ceph.mon.b.smithi107.stdout:Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2019-12-22T22:22:37.138 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.138 INFO:ceph.mon.b.smithi107.stdout:** File Read Latency Histogram By Level [default] ** 2019-12-22T22:22:37.138 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.138 INFO:ceph.mon.b.smithi107.stdout:** Compaction Stats [default] ** 2019-12-22T22:22:37.138 INFO:ceph.mon.b.smithi107.stdout:Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2019-12-22T22:22:37.139 INFO:ceph.mon.b.smithi107.stdout:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2019-12-22T22:22:37.139 INFO:ceph.mon.b.smithi107.stdout: L0 1/0 1.42 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.4 0.00 0.00 1 0.001 0 0 2019-12-22T22:22:37.139 INFO:ceph.mon.b.smithi107.stdout: Sum 1/0 1.42 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.4 0.00 0.00 1 0.001 0 0 2019-12-22T22:22:37.139 INFO:ceph.mon.b.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2019-12-22T22:22:37.139 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.139 INFO:ceph.mon.b.smithi107.stdout:** Compaction Stats [default] ** 2019-12-22T22:22:37.140 INFO:ceph.mon.b.smithi107.stdout:Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2019-12-22T22:22:37.140 INFO:ceph.mon.b.smithi107.stdout:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2019-12-22T22:22:37.141 INFO:ceph.mon.b.smithi107.stdout:User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.4 0.00 0.00 1 0.001 0 0 2019-12-22T22:22:37.141 INFO:ceph.mon.b.smithi107.stdout:Uptime(secs): 0.0 total, 0.0 interval 2019-12-22T22:22:37.141 INFO:ceph.mon.b.smithi107.stdout:Flush(GB): cumulative 0.000, interval 0.000 2019-12-22T22:22:37.142 INFO:ceph.mon.b.smithi107.stdout:AddFile(GB): cumulative 0.000, interval 0.000 2019-12-22T22:22:37.142 INFO:ceph.mon.b.smithi107.stdout:AddFile(Total Files): cumulative 0, interval 0 2019-12-22T22:22:37.142 INFO:ceph.mon.b.smithi107.stdout:AddFile(L0 Files): cumulative 0, interval 0 2019-12-22T22:22:37.142 INFO:ceph.mon.b.smithi107.stdout:AddFile(Keys): cumulative 0, interval 0 2019-12-22T22:22:37.142 INFO:ceph.mon.b.smithi107.stdout:Cumulative compaction: 0.00 GB write, 0.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2019-12-22T22:22:37.142 INFO:ceph.mon.b.smithi107.stdout:Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2019-12-22T22:22:37.143 INFO:ceph.mon.b.smithi107.stdout:Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2019-12-22T22:22:37.143 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.143 INFO:ceph.mon.b.smithi107.stdout:** File Read Latency Histogram By Level [default] ** 2019-12-22T22:22:37.143 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.143 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.524+0000 7f7da21566c0 0 mon.b does not exist in monmap, will attempt to join an existing cluster 2019-12-22T22:22:37.143 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.524+0000 7f7da21566c0 0 using public_addrv v1:172.21.15.107:6789/0 2019-12-22T22:22:37.144 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.525+0000 7f7da21566c0 0 starting mon.b rank -1 at public addrs v1:172.21.15.107:6789/0 at bind addrs v1:172.21.15.107:6789/0 mon_data /var/lib/ceph/mon/ceph-b fsid 147630da-2509-11ea-8284-001a4aab830c 2019-12-22T22:22:37.144 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.527+0000 7f7da21566c0 1 mon.b@-1(???) e0 preinit fsid 147630da-2509-11ea-8284-001a4aab830c 2019-12-22T22:22:37.144 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.552+0000 7f7d87285700 4 rocksdb: [db/db_impl_write.cc:1470] [default] New memtable created with log file: #9. Immutable memtables: 0. 2019-12-22T22:22:37.144 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.144 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.552+0000 7f7d91299700 4 rocksdb: (Original Log Time 2019/12/22-22:22:36.553944) [db/db_impl_compaction_flush.cc:2073] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 2019-12-22T22:22:37.144 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.553+0000 7f7d91299700 4 rocksdb: [db/flush_job.cc:322] [default] [JOB 3] Flushing memtable with next log file: 9 2019-12-22T22:22:37.145 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.145 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.553+0000 7f7d91299700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1577053356554031, "job": 3, "event": "flush_started", "num_memtables": 1, "num_entries": 845, "num_deletes": 31, "total_data_size": 2114753, "memory_usage": 2130632, "flush_reason": "Manual Compaction"} 2019-12-22T22:22:37.145 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.553+0000 7f7d91299700 4 rocksdb: [db/flush_job.cc:351] [default] [JOB 3] Level-0 flush table #10: started 2019-12-22T22:22:37.145 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.559+0000 7f7d8ba8e700 0 mon.b@-1(synchronizing).mds e1 new map 2019-12-22T22:22:37.145 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.559+0000 7f7d8ba8e700 0 mon.b@-1(synchronizing).mds e1 print_map 2019-12-22T22:22:37.145 INFO:ceph.mon.b.smithi107.stdout:e1 2019-12-22T22:22:37.145 INFO:ceph.mon.b.smithi107.stdout:enable_multiple, ever_enabled_multiple: 0,0 2019-12-22T22:22:37.146 INFO:ceph.mon.b.smithi107.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2019-12-22T22:22:37.146 INFO:ceph.mon.b.smithi107.stdout:legacy client fscid: -1 2019-12-22T22:22:37.146 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.146 INFO:ceph.mon.b.smithi107.stdout:No filesystems configured 2019-12-22T22:22:37.146 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.146 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.559+0000 7f7d91299700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1577053356560975, "cf_name": "default", "job": 3, "event": "table_file_creation", "file_number": 10, "file_size": 1961440, "table_properties": {"data_size": 1956050, "index_size": 3351, "filter_size": 1221, "raw_key_size": 16062, "raw_average_key_size": 33, "raw_value_size": 1944198, "raw_average_value_size": 4033, "num_data_blocks": 111, "num_entries": 482, "filter_policy_name": "rocksdb.BuiltinBloomFilter"}} 2019-12-22T22:22:37.146 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.560+0000 7f7d91299700 4 rocksdb: [db/flush_job.cc:392] [default] [JOB 3] Level-0 flush table #10: 1961440 bytes OK 2019-12-22T22:22:37.147 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.560+0000 7f7d91299700 4 rocksdb: (Original Log Time 2019/12/22-22:22:36.561040) [db/memtable_list.cc:380] [default] Level-0 commit table #10 started 2019-12-22T22:22:37.147 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.560+0000 7f7d91299700 4 rocksdb: (Original Log Time 2019/12/22-22:22:36.561263) [db/memtable_list.cc:428] [default] Level-0 commit table #10: memtable #1 done 2019-12-22T22:22:37.147 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.560+0000 7f7d91299700 4 rocksdb: (Original Log Time 2019/12/22-22:22:36.561281) EVENT_LOG_v1 {"time_micros": 1577053356561271, "job": 3, "event": "flush_finished", "output_compression": "NoCompression", "lsm_state": [2, 0, 0, 0, 0, 0, 0], "immutable_memtables": 0} 2019-12-22T22:22:37.147 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.560+0000 7f7d91299700 4 rocksdb: (Original Log Time 2019/12/22-22:22:36.561298) [db/db_impl_compaction_flush.cc:201] [default] Level summary: files[2 0 0 0 0 0 0] max score 0.50 2019-12-22T22:22:37.147 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.147 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.560+0000 7f7d91299700 4 rocksdb: [db/db_impl_files.cc:356] [JOB 3] Try to delete WAL files size 2108831, prev total WAL file size 2679336, number of live WAL files 2. 2019-12-22T22:22:37.148 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.148 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.560+0000 7f7d87285700 4 rocksdb: [db/db_impl_compaction_flush.cc:1403] [default] Manual compaction starting 2019-12-22T22:22:37.148 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.560+0000 7f7d8ba8e700 1 mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2019-12-22T22:22:37.148 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.560+0000 7f7d91a9a700 4 rocksdb: (Original Log Time 2019/12/22-22:22:36.561477) [db/db_impl_compaction_flush.cc:2374] [default] Manual compaction from level-0 to level-6 from 'mgrstat .. 'mgrstat; will stop at (end) 2019-12-22T22:22:37.149 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.149 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.560+0000 7f7d8ba8e700 1 mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2019-12-22T22:22:37.149 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.560+0000 7f7d91a9a700 4 rocksdb: [db/compaction_job.cc:1645] [default] [JOB 4] Compacting 2@0 files to L6, score -1.00 2019-12-22T22:22:37.149 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.560+0000 7f7d91a9a700 4 rocksdb: [db/compaction_job.cc:1649] [default] Compaction start summary: Base version 3 Base level 0, inputs: [10(1915KB) 4(1454B)] 2019-12-22T22:22:37.149 INFO:ceph.mon.b.smithi107.stdout: 2019-12-22T22:22:37.150 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.560+0000 7f7d91a9a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1577053356561558, "job": 4, "event": "compaction_started", "compaction_reason": "ManualCompaction", "files_L0": [10, 4], "score": -1, "input_data_size": 1962894} 2019-12-22T22:22:37.150 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.560+0000 7f7d8ba8e700 1 mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2019-12-22T22:22:37.150 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.560+0000 7f7d8ba8e700 0 mon.b@-1(synchronizing).osd e3 crush map has features 288514050185494528, adjusting msgr requires 2019-12-22T22:22:37.150 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.560+0000 7f7d8ba8e700 0 mon.b@-1(synchronizing).osd e3 crush map has features 288514050185494528, adjusting msgr requires 2019-12-22T22:22:37.151 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.560+0000 7f7d8ba8e700 0 mon.b@-1(synchronizing).osd e3 crush map has features 3314932999778484224, adjusting msgr requires 2019-12-22T22:22:37.151 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.560+0000 7f7d8ba8e700 0 mon.b@-1(synchronizing).osd e3 crush map has features 288514050185494528, adjusting msgr requires 2019-12-22T22:22:37.151 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.561+0000 7f7d8ba8e700 1 mon.b@-1(synchronizing).paxosservice(auth 1..7) refresh upgraded, format 0 -> 3 2019-12-22T22:22:37.151 INFO:ceph.mon.b.smithi107.stdout:2019-12-22T22:22:36.563+0000 7f7d8ba8e700 10 mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete,mon_osdmap_full_prune_interval,mon_osdmap_full_prune_min,mon_osdmap_full_prune_txsize 2019-12-22T22:22:41.854 INFO:teuthology.orchestra.run.smithi107.stdout: 2019-12-22T22:22:41.855 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":3,"fsid":"147630da-2509-11ea-8284-001a4aab830c","modified":"2019-12-22T22:22:36.569843Z","created":"2019-12-22T22:19:57.173839Z","min_mon_release":15,"min_mon_release_name":"octopus","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.91:6789","nonce":0}]},"addr":"172.21.15.91:6789/0","public_addr":"172.21.15.91:6789/0","priority":0,"weight":0},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0}],"quorum":[0,1,2]} 2019-12-22T22:22:41.856 INFO:teuthology.orchestra.run.smithi107.stderr:dumped monmap epoch 3 2019-12-22T22:22:42.383 INFO:tasks.cephadm:Adding mgr.x on smithi107 2019-12-22T22:22:42.383 INFO:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph orchestrator mgr update 2 smithi107=x 2019-12-22T22:22:44.838 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi107,172.21.15.107' (ECDSA) to the list of known hosts. 2019-12-22T22:22:45.838 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi107,172.21.15.107' (ECDSA) to the list of known hosts. 2019-12-22T22:22:48.538 INFO:teuthology.orchestra.run.smithi107.stdout:["(Re)deployed mgr.x on host 'smithi107'"] 2019-12-22T22:22:49.094 INFO:teuthology.orchestra.run.smithi107:mgr.x> sudo /home/ubuntu/cephtest/cephadm logs -f --fsid 147630da-2509-11ea-8284-001a4aab830c --name mgr.x 2019-12-22T22:22:49.100 INFO:tasks.cephadm:Deploying OSDs... 2019-12-22T22:22:49.222 DEBUG:teuthology.orchestra.remote:smithi091:/scratch_devs is 72B 2019-12-22T22:22:49.321 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-22T22:22:49.322 INFO:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2019-12-22T22:22:49.396 INFO:teuthology.orchestra.run.smithi091.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-1’ 2019-12-22T22:22:49.397 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-22T22:22:49.397 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 94973 Links: 1 2019-12-22T22:22:49.397 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-22T22:22:49.397 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:device_t:s0 2019-12-22T22:22:49.398 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2019-12-22 22:20:05.456837476 +0000 2019-12-22T22:22:49.398 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2019-12-22 22:16:20.793560394 +0000 2019-12-22T22:22:49.398 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2019-12-22 22:16:20.793560394 +0000 2019-12-22T22:22:49.398 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: - 2019-12-22T22:22:49.429 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2019-12-22T22:22:49.429 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2019-12-22T22:22:49.430 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes (512 B) copied, 0.000336977 s, 1.5 MB/s 2019-12-22T22:22:49.434 INFO:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2019-12-22T22:22:49.507 INFO:teuthology.orchestra.run.smithi091.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2019-12-22T22:22:49.507 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-22T22:22:49.507 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 110711 Links: 1 2019-12-22T22:22:49.508 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-22T22:22:49.508 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:device_t:s0 2019-12-22T22:22:49.508 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2019-12-22 22:20:05.456837476 +0000 2019-12-22T22:22:49.508 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2019-12-22 22:16:21.276547683 +0000 2019-12-22T22:22:49.509 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2019-12-22 22:16:21.276547683 +0000 2019-12-22T22:22:49.509 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: - 2019-12-22T22:22:49.523 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2019-12-22T22:22:49.524 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2019-12-22T22:22:49.524 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes (512 B) copied, 0.000373422 s, 1.4 MB/s 2019-12-22T22:22:49.526 INFO:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2019-12-22T22:22:49.577 INFO:teuthology.orchestra.run.smithi091.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-3’ 2019-12-22T22:22:49.578 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-22T22:22:49.578 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 105850 Links: 1 2019-12-22T22:22:49.578 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-22T22:22:49.578 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:device_t:s0 2019-12-22T22:22:49.578 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2019-12-22 22:20:05.457837476 +0000 2019-12-22T22:22:49.578 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2019-12-22 22:16:21.740535473 +0000 2019-12-22T22:22:49.579 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2019-12-22 22:16:21.740535473 +0000 2019-12-22T22:22:49.579 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: - 2019-12-22T22:22:49.594 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2019-12-22T22:22:49.595 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2019-12-22T22:22:49.595 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes (512 B) copied, 0.000338248 s, 1.5 MB/s 2019-12-22T22:22:49.597 INFO:teuthology.orchestra.run.smithi091:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2019-12-22T22:22:49.649 INFO:teuthology.orchestra.run.smithi091.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-4’ 2019-12-22T22:22:49.650 INFO:teuthology.orchestra.run.smithi091.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-22T22:22:49.650 INFO:teuthology.orchestra.run.smithi091.stdout:Device: 5h/5d Inode: 105890 Links: 1 2019-12-22T22:22:49.650 INFO:teuthology.orchestra.run.smithi091.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-22T22:22:49.650 INFO:teuthology.orchestra.run.smithi091.stdout:Context: system_u:object_r:device_t:s0 2019-12-22T22:22:49.650 INFO:teuthology.orchestra.run.smithi091.stdout:Access: 2019-12-22 22:20:05.457837476 +0000 2019-12-22T22:22:49.651 INFO:teuthology.orchestra.run.smithi091.stdout:Modify: 2019-12-22 22:16:22.244522209 +0000 2019-12-22T22:22:49.651 INFO:teuthology.orchestra.run.smithi091.stdout:Change: 2019-12-22 22:16:22.244522209 +0000 2019-12-22T22:22:49.651 INFO:teuthology.orchestra.run.smithi091.stdout: Birth: - 2019-12-22T22:22:49.666 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records in 2019-12-22T22:22:49.666 INFO:teuthology.orchestra.run.smithi091.stderr:1+0 records out 2019-12-22T22:22:49.666 INFO:teuthology.orchestra.run.smithi091.stderr:512 bytes (512 B) copied, 0.000314997 s, 1.6 MB/s 2019-12-22T22:22:49.788 DEBUG:teuthology.orchestra.remote:smithi043:/scratch_devs is 72B 2019-12-22T22:22:49.871 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-22T22:22:49.871 INFO:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2019-12-22T22:22:49.955 INFO:teuthology.orchestra.run.smithi043.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-1’ 2019-12-22T22:22:49.955 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-22T22:22:49.955 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 108632 Links: 1 2019-12-22T22:22:49.956 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-22T22:22:49.956 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2019-12-22T22:22:49.956 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2019-12-22 22:22:22.043949079 +0000 2019-12-22T22:22:49.957 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2019-12-22 22:16:27.512299753 +0000 2019-12-22T22:22:49.957 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2019-12-22 22:16:27.512299753 +0000 2019-12-22T22:22:49.957 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: - 2019-12-22T22:22:49.993 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2019-12-22T22:22:49.993 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2019-12-22T22:22:49.993 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes (512 B) copied, 0.000349249 s, 1.5 MB/s 2019-12-22T22:22:50.001 INFO:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2019-12-22T22:22:50.092 INFO:teuthology.orchestra.run.smithi043.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2019-12-22T22:22:50.092 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-22T22:22:50.093 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 108668 Links: 1 2019-12-22T22:22:50.093 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-22T22:22:50.093 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2019-12-22T22:22:50.093 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2019-12-22 22:22:22.043949079 +0000 2019-12-22T22:22:50.094 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2019-12-22 22:16:28.021294129 +0000 2019-12-22T22:22:50.094 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2019-12-22 22:16:28.021294129 +0000 2019-12-22T22:22:50.094 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: - 2019-12-22T22:22:50.122 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2019-12-22T22:22:50.123 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2019-12-22T22:22:50.123 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes (512 B) copied, 0.000328804 s, 1.6 MB/s 2019-12-22T22:22:50.124 INFO:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2019-12-22T22:22:50.206 INFO:teuthology.orchestra.run.smithi043.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-3’ 2019-12-22T22:22:50.206 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-22T22:22:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 110616 Links: 1 2019-12-22T22:22:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-22T22:22:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2019-12-22T22:22:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2019-12-22 22:22:22.043949079 +0000 2019-12-22T22:22:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2019-12-22 22:16:28.519288520 +0000 2019-12-22T22:22:50.207 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2019-12-22 22:16:28.519288520 +0000 2019-12-22T22:22:50.207 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: - 2019-12-22T22:22:50.232 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2019-12-22T22:22:50.232 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2019-12-22T22:22:50.232 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes (512 B) copied, 0.000316833 s, 1.6 MB/s 2019-12-22T22:22:50.237 INFO:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2019-12-22T22:22:50.313 INFO:teuthology.orchestra.run.smithi043.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-4’ 2019-12-22T22:22:50.314 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-22T22:22:50.314 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 110665 Links: 1 2019-12-22T22:22:50.314 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-22T22:22:50.314 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2019-12-22T22:22:50.314 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2019-12-22 22:22:22.043949079 +0000 2019-12-22T22:22:50.315 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2019-12-22 22:16:29.000282942 +0000 2019-12-22T22:22:50.315 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2019-12-22 22:16:29.000282942 +0000 2019-12-22T22:22:50.315 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: - 2019-12-22T22:22:50.341 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2019-12-22T22:22:50.342 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2019-12-22T22:22:50.342 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes (512 B) copied, 0.000317361 s, 1.6 MB/s 2019-12-22T22:22:50.444 DEBUG:teuthology.orchestra.remote:smithi107:/scratch_devs is 72B 2019-12-22T22:22:50.527 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-22T22:22:50.535 INFO:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2019-12-22T22:22:50.591 INFO:teuthology.orchestra.run.smithi107.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-1’ 2019-12-22T22:22:50.592 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-22T22:22:50.592 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 102792 Links: 1 2019-12-22T22:22:50.592 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-22T22:22:50.592 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2019-12-22T22:22:50.593 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2019-12-22 22:22:47.888958087 +0000 2019-12-22T22:22:50.593 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2019-12-22 22:16:21.814074596 +0000 2019-12-22T22:22:50.593 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2019-12-22 22:16:21.814074596 +0000 2019-12-22T22:22:50.593 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2019-12-22T22:22:50.607 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2019-12-22T22:22:50.607 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2019-12-22T22:22:50.607 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes (512 B) copied, 0.000269947 s, 1.9 MB/s 2019-12-22T22:22:50.610 INFO:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2019-12-22T22:22:50.713 INFO:teuthology.orchestra.run.smithi107.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2019-12-22T22:22:50.713 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-22T22:22:50.713 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 102837 Links: 1 2019-12-22T22:22:50.713 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-22T22:22:50.713 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2019-12-22T22:22:50.714 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2019-12-22 22:22:47.888958087 +0000 2019-12-22T22:22:50.714 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2019-12-22 22:16:22.268067380 +0000 2019-12-22T22:22:50.714 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2019-12-22 22:16:22.268067380 +0000 2019-12-22T22:22:50.714 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2019-12-22T22:22:50.731 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2019-12-22T22:22:50.731 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2019-12-22T22:22:50.731 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes (512 B) copied, 0.000316111 s, 1.6 MB/s 2019-12-22T22:22:50.735 INFO:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2019-12-22T22:22:50.791 INFO:teuthology.orchestra.run.smithi107.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-3’ 2019-12-22T22:22:50.792 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-22T22:22:50.792 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 110116 Links: 1 2019-12-22T22:22:50.792 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-22T22:22:50.792 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2019-12-22T22:22:50.792 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2019-12-22 22:22:47.889958078 +0000 2019-12-22T22:22:50.792 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2019-12-22 22:16:22.773059352 +0000 2019-12-22T22:22:50.793 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2019-12-22 22:16:22.773059352 +0000 2019-12-22T22:22:50.793 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2019-12-22T22:22:50.806 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2019-12-22T22:22:50.806 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2019-12-22T22:22:50.806 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes (512 B) copied, 0.000269342 s, 1.9 MB/s 2019-12-22T22:22:50.809 INFO:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2019-12-22T22:22:50.856 INFO:teuthology.orchestra.run.smithi107.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-4’ 2019-12-22T22:22:50.857 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-22T22:22:50.857 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 102903 Links: 1 2019-12-22T22:22:50.857 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-22T22:22:50.857 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2019-12-22T22:22:50.857 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2019-12-22 22:22:47.889958078 +0000 2019-12-22T22:22:50.857 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2019-12-22 22:16:23.276051356 +0000 2019-12-22T22:22:50.857 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2019-12-22 22:16:23.276051356 +0000 2019-12-22T22:22:50.858 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2019-12-22T22:22:50.869 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2019-12-22T22:22:50.869 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2019-12-22T22:22:50.869 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes (512 B) copied, 0.000272638 s, 1.9 MB/s 2019-12-22T22:22:50.871 INFO:tasks.cephadm:Deploying osd.0 on smithi091 with /dev/vg_nvme/lv_1... 2019-12-22T22:22:50.872 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2019-12-22T22:22:53.129 INFO:teuthology.orchestra.run.smithi091.stderr: stderr: /usr/sbin/udevadm: error while loading shared libraries: libsystemd-shared-239.so: cannot open shared object file: No such file or directory 2019-12-22T22:22:53.129 INFO:teuthology.orchestra.run.smithi091.stderr:--> Zapping: /dev/vg_nvme/lv_1 2019-12-22T22:22:53.129 INFO:teuthology.orchestra.run.smithi091.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2019-12-22T22:22:53.130 INFO:teuthology.orchestra.run.smithi091.stderr: stderr: 10+0 records in 2019-12-22T22:22:53.130 INFO:teuthology.orchestra.run.smithi091.stderr:10+0 records out 2019-12-22T22:22:53.130 INFO:teuthology.orchestra.run.smithi091.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0147201 s, 712 MB/s 2019-12-22T22:22:53.130 INFO:teuthology.orchestra.run.smithi091.stderr:--> Zapping successful for: 2019-12-22T22:22:53.730 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph orchestrator osd create smithi091:/dev/vg_nvme/lv_1 2019-12-22T22:22:56.095 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi091,172.21.15.91' (ECDSA) to the list of known hosts. 2019-12-22T22:23:03.847 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi091,172.21.15.91' (ECDSA) to the list of known hosts. 2019-12-22T22:23:07.348 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi091,172.21.15.91' (ECDSA) to the list of known hosts. 2019-12-22T22:23:09.952 INFO:teuthology.orchestra.run.smithi091.stdout:Created osd(s) on host 'smithi091' 2019-12-22T22:23:10.560 INFO:teuthology.orchestra.run.smithi091:osd.0> sudo /home/ubuntu/cephtest/cephadm logs -f --fsid 147630da-2509-11ea-8284-001a4aab830c --name osd.0 2019-12-22T22:23:10.565 INFO:tasks.cephadm:Deploying osd.1 on smithi091 with /dev/vg_nvme/lv_2... 2019-12-22T22:23:10.565 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2019-12-22T22:23:10.946 INFO:ceph.osd.0.smithi091.stderr:Error: no container with name or ID ceph-147630da-2509-11ea-8284-001a4aab830c-osd.0 found: no such container 2019-12-22T22:23:13.135 INFO:teuthology.orchestra.run.smithi091.stderr: stderr: /usr/sbin/udevadm: error while loading shared libraries: libsystemd-shared-239.so: cannot open shared object file: No such file or directory 2019-12-22T22:23:13.136 INFO:teuthology.orchestra.run.smithi091.stderr:--> Zapping: /dev/vg_nvme/lv_2 2019-12-22T22:23:13.136 INFO:teuthology.orchestra.run.smithi091.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2019-12-22T22:23:13.136 INFO:teuthology.orchestra.run.smithi091.stderr: stderr: 10+0 records in 2019-12-22T22:23:13.136 INFO:teuthology.orchestra.run.smithi091.stderr:10+0 records out 2019-12-22T22:23:13.136 INFO:teuthology.orchestra.run.smithi091.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0159988 s, 655 MB/s 2019-12-22T22:23:13.137 INFO:teuthology.orchestra.run.smithi091.stderr:--> Zapping successful for: 2019-12-22T22:23:13.676 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph orchestrator osd create smithi091:/dev/vg_nvme/lv_2 2019-12-22T22:23:16.106 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi091,172.21.15.91' (ECDSA) to the list of known hosts. 2019-12-22T22:23:23.861 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi091,172.21.15.91' (ECDSA) to the list of known hosts. 2019-12-22T22:23:27.129 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi091,172.21.15.91' (ECDSA) to the list of known hosts. 2019-12-22T22:23:29.854 INFO:teuthology.orchestra.run.smithi091.stdout:Created osd(s) on host 'smithi091' 2019-12-22T22:23:30.531 INFO:teuthology.orchestra.run.smithi091:osd.1> sudo /home/ubuntu/cephtest/cephadm logs -f --fsid 147630da-2509-11ea-8284-001a4aab830c --name osd.1 2019-12-22T22:23:30.537 INFO:tasks.cephadm:Deploying osd.2 on smithi091 with /dev/vg_nvme/lv_3... 2019-12-22T22:23:30.537 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2019-12-22T22:23:30.857 INFO:ceph.osd.1.smithi091.stderr:Error: no container with name or ID ceph-147630da-2509-11ea-8284-001a4aab830c-osd.1 found: no such container 2019-12-22T22:23:32.939 INFO:teuthology.orchestra.run.smithi091.stderr: stderr: /usr/sbin/udevadm: error while loading shared libraries: libsystemd-shared-239.so: cannot open shared object file: No such file or directory 2019-12-22T22:23:32.939 INFO:teuthology.orchestra.run.smithi091.stderr:--> Zapping: /dev/vg_nvme/lv_3 2019-12-22T22:23:32.939 INFO:teuthology.orchestra.run.smithi091.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2019-12-22T22:23:32.939 INFO:teuthology.orchestra.run.smithi091.stderr: stderr: 10+0 records in 2019-12-22T22:23:32.940 INFO:teuthology.orchestra.run.smithi091.stderr:10+0 records out 2019-12-22T22:23:32.940 INFO:teuthology.orchestra.run.smithi091.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0195577 s, 536 MB/s 2019-12-22T22:23:32.940 INFO:teuthology.orchestra.run.smithi091.stderr:--> Zapping successful for: 2019-12-22T22:23:33.601 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph orchestrator osd create smithi091:/dev/vg_nvme/lv_3 2019-12-22T22:23:35.883 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi091,172.21.15.91' (ECDSA) to the list of known hosts. 2019-12-22T22:23:43.897 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi091,172.21.15.91' (ECDSA) to the list of known hosts. 2019-12-22T22:23:47.142 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi091,172.21.15.91' (ECDSA) to the list of known hosts. 2019-12-22T22:23:49.838 INFO:teuthology.orchestra.run.smithi091.stdout:Created osd(s) on host 'smithi091' 2019-12-22T22:23:50.534 INFO:teuthology.orchestra.run.smithi091:osd.2> sudo /home/ubuntu/cephtest/cephadm logs -f --fsid 147630da-2509-11ea-8284-001a4aab830c --name osd.2 2019-12-22T22:23:50.554 INFO:tasks.cephadm:Deploying osd.3 on smithi091 with /dev/vg_nvme/lv_4... 2019-12-22T22:23:50.554 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2019-12-22T22:23:50.902 INFO:ceph.osd.2.smithi091.stderr:Error: no container with name or ID ceph-147630da-2509-11ea-8284-001a4aab830c-osd.2 found: no such container 2019-12-22T22:23:53.020 INFO:teuthology.orchestra.run.smithi091.stderr: stderr: /usr/sbin/udevadm: error while loading shared libraries: libsystemd-shared-239.so: cannot open shared object file: No such file or directory 2019-12-22T22:23:53.020 INFO:teuthology.orchestra.run.smithi091.stderr:--> Zapping: /dev/vg_nvme/lv_4 2019-12-22T22:23:53.021 INFO:teuthology.orchestra.run.smithi091.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2019-12-22T22:23:53.021 INFO:teuthology.orchestra.run.smithi091.stderr: stderr: 10+0 records in 2019-12-22T22:23:53.021 INFO:teuthology.orchestra.run.smithi091.stderr:10+0 records out 2019-12-22T22:23:53.021 INFO:teuthology.orchestra.run.smithi091.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0143481 s, 731 MB/s 2019-12-22T22:23:53.021 INFO:teuthology.orchestra.run.smithi091.stderr:--> Zapping successful for: 2019-12-22T22:23:53.604 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph orchestrator osd create smithi091:/dev/vg_nvme/lv_4 2019-12-22T22:23:56.151 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi091,172.21.15.91' (ECDSA) to the list of known hosts. 2019-12-22T22:24:03.908 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi091,172.21.15.91' (ECDSA) to the list of known hosts. 2019-12-22T22:24:07.162 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi091,172.21.15.91' (ECDSA) to the list of known hosts. 2019-12-22T22:24:09.883 INFO:teuthology.orchestra.run.smithi091.stdout:Created osd(s) on host 'smithi091' 2019-12-22T22:24:10.624 INFO:teuthology.orchestra.run.smithi091:osd.3> sudo /home/ubuntu/cephtest/cephadm logs -f --fsid 147630da-2509-11ea-8284-001a4aab830c --name osd.3 2019-12-22T22:24:10.629 INFO:tasks.cephadm:Deploying osd.4 on smithi107 with /dev/vg_nvme/lv_1... 2019-12-22T22:24:10.629 INFO:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2019-12-22T22:24:11.043 INFO:ceph.osd.3.smithi091.stderr:Error: no container with name or ID ceph-147630da-2509-11ea-8284-001a4aab830c-osd.3 found: no such container 2019-12-22T22:24:12.819 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: /usr/sbin/udevadm: error while loading shared libraries: libsystemd-shared-239.so: cannot open shared object file: No such file or directory 2019-12-22T22:24:12.819 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_1 2019-12-22T22:24:12.820 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2019-12-22T22:24:12.820 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2019-12-22T22:24:12.820 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2019-12-22T22:24:12.820 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0167994 s, 624 MB/s 2019-12-22T22:24:12.820 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2019-12-22T22:24:13.394 INFO:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph orchestrator osd create smithi107:/dev/vg_nvme/lv_1 2019-12-22T22:24:15.667 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi107,172.21.15.107' (ECDSA) to the list of known hosts. 2019-12-22T22:24:23.671 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi107,172.21.15.107' (ECDSA) to the list of known hosts. 2019-12-22T22:24:27.173 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi107,172.21.15.107' (ECDSA) to the list of known hosts. 2019-12-22T22:24:29.551 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) on host 'smithi107' 2019-12-22T22:24:30.299 INFO:teuthology.orchestra.run.smithi107:osd.4> sudo /home/ubuntu/cephtest/cephadm logs -f --fsid 147630da-2509-11ea-8284-001a4aab830c --name osd.4 2019-12-22T22:24:30.305 INFO:tasks.cephadm:Deploying osd.5 on smithi107 with /dev/vg_nvme/lv_2... 2019-12-22T22:24:30.305 INFO:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2019-12-22T22:24:30.807 INFO:ceph.osd.4.smithi107.stderr:Error: no container with name or ID ceph-147630da-2509-11ea-8284-001a4aab830c-osd.4 found: no such container 2019-12-22T22:24:32.981 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: /usr/sbin/udevadm: error while loading shared libraries: libsystemd-shared-239.so: cannot open shared object file: No such file or directory 2019-12-22T22:24:32.981 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_2 2019-12-22T22:24:32.981 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2019-12-22T22:24:32.981 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2019-12-22T22:24:32.982 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2019-12-22T22:24:32.982 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0214576 s, 489 MB/s 2019-12-22T22:24:32.982 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2019-12-22T22:24:33.579 INFO:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph orchestrator osd create smithi107:/dev/vg_nvme/lv_2 2019-12-22T22:24:35.928 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi107,172.21.15.107' (ECDSA) to the list of known hosts. 2019-12-22T22:24:43.934 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi107,172.21.15.107' (ECDSA) to the list of known hosts. 2019-12-22T22:24:47.437 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi107,172.21.15.107' (ECDSA) to the list of known hosts. 2019-12-22T22:24:49.861 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) on host 'smithi107' 2019-12-22T22:24:50.526 INFO:teuthology.orchestra.run.smithi107:osd.5> sudo /home/ubuntu/cephtest/cephadm logs -f --fsid 147630da-2509-11ea-8284-001a4aab830c --name osd.5 2019-12-22T22:24:50.534 INFO:tasks.cephadm:Deploying osd.6 on smithi107 with /dev/vg_nvme/lv_3... 2019-12-22T22:24:50.534 INFO:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2019-12-22T22:24:50.945 INFO:ceph.osd.5.smithi107.stderr:Error: no container with name or ID ceph-147630da-2509-11ea-8284-001a4aab830c-osd.5 found: no such container 2019-12-22T22:24:53.226 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: /usr/sbin/udevadm: error while loading shared libraries: libsystemd-shared-239.so: cannot open shared object file: No such file or directory 2019-12-22T22:24:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_3 2019-12-22T22:24:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2019-12-22T22:24:53.226 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2019-12-22T22:24:53.226 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2019-12-22T22:24:53.227 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0170175 s, 616 MB/s 2019-12-22T22:24:53.227 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2019-12-22T22:24:53.822 INFO:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph orchestrator osd create smithi107:/dev/vg_nvme/lv_3 2019-12-22T22:24:56.189 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi107,172.21.15.107' (ECDSA) to the list of known hosts. 2019-12-22T22:25:04.193 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi107,172.21.15.107' (ECDSA) to the list of known hosts. 2019-12-22T22:25:07.696 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi107,172.21.15.107' (ECDSA) to the list of known hosts. 2019-12-22T22:25:09.998 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) on host 'smithi107' 2019-12-22T22:25:10.780 INFO:teuthology.orchestra.run.smithi107:osd.6> sudo /home/ubuntu/cephtest/cephadm logs -f --fsid 147630da-2509-11ea-8284-001a4aab830c --name osd.6 2019-12-22T22:25:10.786 INFO:tasks.cephadm:Deploying osd.7 on smithi107 with /dev/vg_nvme/lv_4... 2019-12-22T22:25:10.786 INFO:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2019-12-22T22:25:11.265 INFO:ceph.osd.6.smithi107.stderr:Error: no container with name or ID ceph-147630da-2509-11ea-8284-001a4aab830c-osd.6 found: no such container 2019-12-22T22:25:13.585 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: /usr/sbin/udevadm: error while loading shared libraries: libsystemd-shared-239.so: cannot open shared object file: No such file or directory 2019-12-22T22:25:13.586 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_4 2019-12-22T22:25:13.586 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2019-12-22T22:25:13.586 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2019-12-22T22:25:13.587 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2019-12-22T22:25:13.587 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0207584 s, 505 MB/s 2019-12-22T22:25:13.588 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2019-12-22T22:25:14.197 INFO:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph orchestrator osd create smithi107:/dev/vg_nvme/lv_4 2019-12-22T22:25:16.700 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi107,172.21.15.107' (ECDSA) to the list of known hosts. 2019-12-22T22:25:24.704 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi107,172.21.15.107' (ECDSA) to the list of known hosts. 2019-12-22T22:25:28.208 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi107,172.21.15.107' (ECDSA) to the list of known hosts. 2019-12-22T22:25:30.632 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) on host 'smithi107' 2019-12-22T22:25:31.305 INFO:teuthology.orchestra.run.smithi107:osd.7> sudo /home/ubuntu/cephtest/cephadm logs -f --fsid 147630da-2509-11ea-8284-001a4aab830c --name osd.7 2019-12-22T22:25:31.309 INFO:tasks.cephadm:Deploying osd.8 on smithi043 with /dev/vg_nvme/lv_1... 2019-12-22T22:25:31.310 INFO:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2019-12-22T22:25:31.750 INFO:ceph.osd.7.smithi107.stderr:Error: no container with name or ID ceph-147630da-2509-11ea-8284-001a4aab830c-osd.7 found: no such container 2019-12-22T22:25:33.981 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: /usr/sbin/udevadm: error while loading shared libraries: libsystemd-shared-239.so: cannot open shared object file: No such file or directory 2019-12-22T22:25:33.981 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_1 2019-12-22T22:25:33.981 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2019-12-22T22:25:33.982 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2019-12-22T22:25:33.982 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2019-12-22T22:25:33.982 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0174121 s, 602 MB/s 2019-12-22T22:25:33.982 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2019-12-22T22:25:34.644 INFO:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph orchestrator osd create smithi043:/dev/vg_nvme/lv_1 2019-12-22T22:25:36.964 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi043,172.21.15.43' (ECDSA) to the list of known hosts. 2019-12-22T22:25:44.465 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi043,172.21.15.43' (ECDSA) to the list of known hosts. 2019-12-22T22:25:47.967 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi043,172.21.15.43' (ECDSA) to the list of known hosts. 2019-12-22T22:25:50.783 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) on host 'smithi043' 2019-12-22T22:25:51.615 INFO:teuthology.orchestra.run.smithi043:osd.8> sudo /home/ubuntu/cephtest/cephadm logs -f --fsid 147630da-2509-11ea-8284-001a4aab830c --name osd.8 2019-12-22T22:25:51.620 INFO:tasks.cephadm:Deploying osd.9 on smithi043 with /dev/vg_nvme/lv_2... 2019-12-22T22:25:51.620 INFO:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2019-12-22T22:25:52.668 INFO:ceph.osd.8.smithi043.stdout:2019-12-22T22:25:52.607+0000 7f437c40fec0 -1 Falling back to public interface 2019-12-22T22:25:53.166 INFO:ceph.osd.8.smithi043.stdout:2019-12-22T22:25:53.160+0000 7f437c40fec0 -1 osd.8 0 log_to_monitors {default=true} 2019-12-22T22:25:53.882 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: /usr/sbin/udevadm: error while loading shared libraries: libsystemd-shared-239.so: cannot open shared object file: No such file or directory 2019-12-22T22:25:53.882 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_2 2019-12-22T22:25:53.883 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2019-12-22T22:25:53.883 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2019-12-22T22:25:53.883 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2019-12-22T22:25:53.883 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0227169 s, 462 MB/s 2019-12-22T22:25:53.883 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2019-12-22T22:25:54.432 INFO:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph orchestrator osd create smithi043:/dev/vg_nvme/lv_2 2019-12-22T22:25:55.169 INFO:ceph.osd.8.smithi043.stdout:2019-12-22T22:25:54.967+0000 7f43611af700 -1 osd.8 0 waiting for initial osdmap 2019-12-22T22:25:56.724 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi043,172.21.15.43' (ECDSA) to the list of known hosts. 2019-12-22T22:26:04.226 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi043,172.21.15.43' (ECDSA) to the list of known hosts. 2019-12-22T22:26:07.728 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi043,172.21.15.43' (ECDSA) to the list of known hosts. 2019-12-22T22:26:10.601 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) on host 'smithi043' 2019-12-22T22:26:11.537 INFO:teuthology.orchestra.run.smithi043:osd.9> sudo /home/ubuntu/cephtest/cephadm logs -f --fsid 147630da-2509-11ea-8284-001a4aab830c --name osd.9 2019-12-22T22:26:11.542 INFO:tasks.cephadm:Deploying osd.10 on smithi043 with /dev/vg_nvme/lv_3... 2019-12-22T22:26:11.542 INFO:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2019-12-22T22:26:12.080 INFO:ceph.osd.9.smithi043.stdout:2019-12-22T22:26:11.948+0000 7f863f125ec0 1 -- 172.21.15.43:0/1150032114 >> v1:172.21.15.43:6789/0 conn(0x562dd434c400 legacy=0x562dd433a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2019-12-22T22:26:12.578 INFO:ceph.osd.9.smithi043.stdout:2019-12-22T22:26:12.460+0000 7f863f125ec0 -1 Falling back to public interface 2019-12-22T22:26:13.079 INFO:ceph.osd.9.smithi043.stdout:2019-12-22T22:26:13.039+0000 7f863f125ec0 -1 osd.9 0 log_to_monitors {default=true} 2019-12-22T22:26:13.877 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: /usr/sbin/udevadm: error while loading shared libraries: libsystemd-shared-239.so: cannot open shared object file: No such file or directory 2019-12-22T22:26:13.877 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_3 2019-12-22T22:26:13.878 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2019-12-22T22:26:13.878 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2019-12-22T22:26:13.878 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2019-12-22T22:26:13.878 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0101047 s, 1.0 GB/s 2019-12-22T22:26:13.878 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2019-12-22T22:26:14.463 INFO:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph orchestrator osd create smithi043:/dev/vg_nvme/lv_3 2019-12-22T22:26:15.080 INFO:ceph.osd.9.smithi043.stdout:2019-12-22T22:26:14.966+0000 7f8623ec5700 -1 osd.9 0 waiting for initial osdmap 2019-12-22T22:26:16.983 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi043,172.21.15.43' (ECDSA) to the list of known hosts. 2019-12-22T22:26:24.737 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi043,172.21.15.43' (ECDSA) to the list of known hosts. 2019-12-22T22:26:28.239 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi043,172.21.15.43' (ECDSA) to the list of known hosts. 2019-12-22T22:26:30.752 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) on host 'smithi043' 2019-12-22T22:26:31.393 INFO:teuthology.orchestra.run.smithi043:osd.10> sudo /home/ubuntu/cephtest/cephadm logs -f --fsid 147630da-2509-11ea-8284-001a4aab830c --name osd.10 2019-12-22T22:26:31.398 INFO:tasks.cephadm:Deploying osd.11 on smithi043 with /dev/vg_nvme/lv_4... 2019-12-22T22:26:31.398 INFO:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2019-12-22T22:26:31.686 INFO:ceph.osd.10.smithi043.stderr:Error: no container with name or ID ceph-147630da-2509-11ea-8284-001a4aab830c-osd.10 found: no such container 2019-12-22T22:26:34.177 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: /usr/sbin/udevadm: error while loading shared libraries: libsystemd-shared-239.so: cannot open shared object file: No such file or directory 2019-12-22T22:26:34.178 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_4 2019-12-22T22:26:34.178 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2019-12-22T22:26:34.178 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2019-12-22T22:26:34.178 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2019-12-22T22:26:34.178 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.014914 s, 703 MB/s 2019-12-22T22:26:34.178 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2019-12-22T22:26:34.776 INFO:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph orchestrator osd create smithi043:/dev/vg_nvme/lv_4 2019-12-22T22:26:37.244 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi043,172.21.15.43' (ECDSA) to the list of known hosts. 2019-12-22T22:26:45.248 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi043,172.21.15.43' (ECDSA) to the list of known hosts. 2019-12-22T22:26:48.500 INFO:ceph.mgr.y.smithi091.stdout:Warning: Permanently added 'smithi043,172.21.15.43' (ECDSA) to the list of known hosts. 2019-12-22T22:26:51.079 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) on host 'smithi043' 2019-12-22T22:26:51.852 INFO:teuthology.orchestra.run.smithi043:osd.11> sudo /home/ubuntu/cephtest/cephadm logs -f --fsid 147630da-2509-11ea-8284-001a4aab830c --name osd.11 2019-12-22T22:26:51.857 INFO:tasks.cephadm:Distributing config and client.admin keyring... 2019-12-22T22:26:51.857 INFO:teuthology.orchestra.run.smithi091:> sudo mkdir -p /etc/ceph 2019-12-22T22:26:52.036 INFO:teuthology.orchestra.run.smithi091:> sudo sh -c 'cat > /etc/ceph/ceph.conf' 2019-12-22T22:26:52.135 INFO:teuthology.orchestra.run.smithi091:> sudo sh -c 'cat > /etc/ceph/ceph.client.admin.keyring' 2019-12-22T22:26:52.212 INFO:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph 2019-12-22T22:26:52.318 INFO:ceph.osd.11.smithi043.stderr:Error: no container with name or ID ceph-147630da-2509-11ea-8284-001a4aab830c-osd.11 found: no such container 2019-12-22T22:26:52.358 INFO:teuthology.orchestra.run.smithi043:> sudo sh -c 'cat > /etc/ceph/ceph.conf' 2019-12-22T22:26:52.509 INFO:teuthology.orchestra.run.smithi043:> sudo sh -c 'cat > /etc/ceph/ceph.client.admin.keyring' 2019-12-22T22:26:52.607 INFO:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph 2019-12-22T22:26:52.775 INFO:teuthology.orchestra.run.smithi107:> sudo sh -c 'cat > /etc/ceph/ceph.conf' 2019-12-22T22:26:52.869 INFO:teuthology.orchestra.run.smithi107:> sudo sh -c 'cat > /etc/ceph/ceph.client.admin.keyring' 2019-12-22T22:26:52.962 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd dump --format=json 2019-12-22T22:26:55.181 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:26:55.181 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":61,"fsid":"147630da-2509-11ea-8284-001a4aab830c","created":"2019-12-22T22:20:00.154953+0000","modified":"2019-12-22T22:26:54.961825+0000","last_up_change":"2019-12-22T22:26:35.964857+0000","last_in_change":"2019-12-22T22:26:35.964857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":25,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":12,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"4520fba2-2eb9-4a60-8945-d2d3a0e5128d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.91:6803","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":1},{"type":"v1","addr":"172.21.15.91:6805","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":1},{"type":"v1","addr":"172.21.15.91:6809","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":1},{"type":"v1","addr":"172.21.15.91:6807","nonce":1}]},"public_addr":"172.21.15.91:6803/1","cluster_addr":"172.21.15.91:6805/1","heartbeat_back_addr":"172.21.15.91:6809/1","heartbeat_front_addr":"172.21.15.91:6807/1","state":["exists","up"]},{"osd":1,"uuid":"ff7b9947-09d3-4363-9073-f074112a0eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.91:6811","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":1},{"type":"v1","addr":"172.21.15.91:6813","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":1},{"type":"v1","addr":"172.21.15.91:6817","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":1},{"type":"v1","addr":"172.21.15.91:6815","nonce":1}]},"public_addr":"172.21.15.91:6811/1","cluster_addr":"172.21.15.91:6813/1","heartbeat_back_addr":"172.21.15.91:6817/1","heartbeat_front_addr":"172.21.15.91:6815/1","state":["exists","up"]},{"osd":2,"uuid":"ccdb35fb-e575-4f63-8f4a-3dc721d55812","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.91:6819","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":1},{"type":"v1","addr":"172.21.15.91:6821","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":1},{"type":"v1","addr":"172.21.15.91:6825","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6822","nonce":1},{"type":"v1","addr":"172.21.15.91:6823","nonce":1}]},"public_addr":"172.21.15.91:6819/1","cluster_addr":"172.21.15.91:6821/1","heartbeat_back_addr":"172.21.15.91:6825/1","heartbeat_front_addr":"172.21.15.91:6823/1","state":["exists","up"]},{"osd":3,"uuid":"5eac4685-b5d5-41e5-84d7-11702f96d78e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.91:6827","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":1},{"type":"v1","addr":"172.21.15.91:6829","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":1},{"type":"v1","addr":"172.21.15.91:6833","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":1},{"type":"v1","addr":"172.21.15.91:6831","nonce":1}]},"public_addr":"172.21.15.91:6827/1","cluster_addr":"172.21.15.91:6829/1","heartbeat_back_addr":"172.21.15.91:6833/1","heartbeat_front_addr":"172.21.15.91:6831/1","state":["exists","up"]},{"osd":4,"uuid":"0c8c0189-2dd0-4a79-9e1f-8533d73b4480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6801","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1},{"type":"v1","addr":"172.21.15.107:6803","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1},{"type":"v1","addr":"172.21.15.107:6807","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1},{"type":"v1","addr":"172.21.15.107:6805","nonce":1}]},"public_addr":"172.21.15.107:6801/1","cluster_addr":"172.21.15.107:6803/1","heartbeat_back_addr":"172.21.15.107:6807/1","heartbeat_front_addr":"172.21.15.107:6805/1","state":["exists","up"]},{"osd":5,"uuid":"3a1924d5-2253-435f-a199-2286ef771c27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6809","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1},{"type":"v1","addr":"172.21.15.107:6811","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1},{"type":"v1","addr":"172.21.15.107:6815","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1},{"type":"v1","addr":"172.21.15.107:6813","nonce":1}]},"public_addr":"172.21.15.107:6809/1","cluster_addr":"172.21.15.107:6811/1","heartbeat_back_addr":"172.21.15.107:6815/1","heartbeat_front_addr":"172.21.15.107:6813/1","state":["exists","up"]},{"osd":6,"uuid":"56e1405c-eeda-4e92-b8de-b330fe2b9a7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6817","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1},{"type":"v1","addr":"172.21.15.107:6819","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1},{"type":"v1","addr":"172.21.15.107:6823","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1},{"type":"v1","addr":"172.21.15.107:6821","nonce":1}]},"public_addr":"172.21.15.107:6817/1","cluster_addr":"172.21.15.107:6819/1","heartbeat_back_addr":"172.21.15.107:6823/1","heartbeat_front_addr":"172.21.15.107:6821/1","state":["exists","up"]},{"osd":7,"uuid":"9a55297d-6a61-444a-a3b1-66293cb3dfe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6825","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1},{"type":"v1","addr":"172.21.15.107:6827","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1},{"type":"v1","addr":"172.21.15.107:6831","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1},{"type":"v1","addr":"172.21.15.107:6829","nonce":1}]},"public_addr":"172.21.15.107:6825/1","cluster_addr":"172.21.15.107:6827/1","heartbeat_back_addr":"172.21.15.107:6831/1","heartbeat_front_addr":"172.21.15.107:6829/1","state":["exists","up"]},{"osd":8,"uuid":"71cd6cef-0dd6-4744-bd1e-d407cc9f4e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6801","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1},{"type":"v1","addr":"172.21.15.43:6803","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1},{"type":"v1","addr":"172.21.15.43:6807","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1},{"type":"v1","addr":"172.21.15.43:6805","nonce":1}]},"public_addr":"172.21.15.43:6801/1","cluster_addr":"172.21.15.43:6803/1","heartbeat_back_addr":"172.21.15.43:6807/1","heartbeat_front_addr":"172.21.15.43:6805/1","state":["exists","up"]},{"osd":9,"uuid":"7c1f2ce9-60f1-49ce-8bdb-1608d4a5b47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6809","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1},{"type":"v1","addr":"172.21.15.43:6811","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1},{"type":"v1","addr":"172.21.15.43:6815","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1},{"type":"v1","addr":"172.21.15.43:6813","nonce":1}]},"public_addr":"172.21.15.43:6809/1","cluster_addr":"172.21.15.43:6811/1","heartbeat_back_addr":"172.21.15.43:6815/1","heartbeat_front_addr":"172.21.15.43:6813/1","state":["exists","up"]},{"osd":10,"uuid":"81aebf87-7a3a-4dc2-af79-2f21da7a7832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6817","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1},{"type":"v1","addr":"172.21.15.43:6819","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1},{"type":"v1","addr":"172.21.15.43:6823","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1},{"type":"v1","addr":"172.21.15.43:6821","nonce":1}]},"public_addr":"172.21.15.43:6817/1","cluster_addr":"172.21.15.43:6819/1","heartbeat_back_addr":"172.21.15.43:6823/1","heartbeat_front_addr":"172.21.15.43:6821/1","state":["exists","up"]},{"osd":11,"uuid":"d025b7a0-a2cb-4769-9908-5d1a93507c1f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:15.319727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:36.199506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:25:54.536613+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:14.187705+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:34.231175+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:25:54.078098+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:15.435592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:35.215727+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:54.884875+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:14.037308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:34.562350+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.91:6801/1":"2019-12-23T22:20:16.666965+0000","172.21.15.91:6800/1":"2019-12-23T22:20:16.666965+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-22T22:26:55.682 INFO:tasks.cephadm.ceph_manager.ceph:[] 2019-12-22T22:26:55.682 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2019-12-22T22:26:55.682 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2019-12-22T22:26:55.682 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph mgr dump --format=json 2019-12-22T22:26:57.800 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:26:58.345 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":11,"active_gid":14120,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6800","nonce":1},{"type":"v1","addr":"172.21.15.91:6801","nonce":1}]},"active_addr":"172.21.15.91:6801/1","active_change":"2019-12-22T22:20:44.779304+0000","active_mgr_features":4540138292836696063,"available":true,"standbys":[{"gid":34103,"name":"x","mgr_features":4540138292836696063,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"ca_bundle":{"name":"ca_bundle","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_location":{"name":"server_location","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"1","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"inventory_cache_timeout":{"name":"inventory_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"service_cache_timeout":{"name":"service_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"str","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"str","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"str","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["ansible","cephadm","deepsea","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"ca_bundle":{"name":"ca_bundle","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_location":{"name":"server_location","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"1","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"inventory_cache_timeout":{"name":"inventory_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"service_cache_timeout":{"name":"service_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"str","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"str","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"str","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["ansible","cephadm","deepsea","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://smithi091.front.sepia.ceph.com:8443/"},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator_cli","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":3,"active_clients":[]}} 2019-12-22T22:26:58.351 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2019-12-22T22:26:58.351 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2019-12-22T22:26:58.351 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd dump --format=json 2019-12-22T22:27:00.566 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:27:00.566 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":63,"fsid":"147630da-2509-11ea-8284-001a4aab830c","created":"2019-12-22T22:20:00.154953+0000","modified":"2019-12-22T22:26:56.979964+0000","last_up_change":"2019-12-22T22:26:55.970471+0000","last_in_change":"2019-12-22T22:26:55.970471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":25,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":12,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"4520fba2-2eb9-4a60-8945-d2d3a0e5128d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.91:6803","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":1},{"type":"v1","addr":"172.21.15.91:6805","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":1},{"type":"v1","addr":"172.21.15.91:6809","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":1},{"type":"v1","addr":"172.21.15.91:6807","nonce":1}]},"public_addr":"172.21.15.91:6803/1","cluster_addr":"172.21.15.91:6805/1","heartbeat_back_addr":"172.21.15.91:6809/1","heartbeat_front_addr":"172.21.15.91:6807/1","state":["exists","up"]},{"osd":1,"uuid":"ff7b9947-09d3-4363-9073-f074112a0eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.91:6811","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":1},{"type":"v1","addr":"172.21.15.91:6813","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":1},{"type":"v1","addr":"172.21.15.91:6817","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":1},{"type":"v1","addr":"172.21.15.91:6815","nonce":1}]},"public_addr":"172.21.15.91:6811/1","cluster_addr":"172.21.15.91:6813/1","heartbeat_back_addr":"172.21.15.91:6817/1","heartbeat_front_addr":"172.21.15.91:6815/1","state":["exists","up"]},{"osd":2,"uuid":"ccdb35fb-e575-4f63-8f4a-3dc721d55812","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.91:6819","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":1},{"type":"v1","addr":"172.21.15.91:6821","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":1},{"type":"v1","addr":"172.21.15.91:6825","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6822","nonce":1},{"type":"v1","addr":"172.21.15.91:6823","nonce":1}]},"public_addr":"172.21.15.91:6819/1","cluster_addr":"172.21.15.91:6821/1","heartbeat_back_addr":"172.21.15.91:6825/1","heartbeat_front_addr":"172.21.15.91:6823/1","state":["exists","up"]},{"osd":3,"uuid":"5eac4685-b5d5-41e5-84d7-11702f96d78e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.91:6827","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":1},{"type":"v1","addr":"172.21.15.91:6829","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":1},{"type":"v1","addr":"172.21.15.91:6833","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":1},{"type":"v1","addr":"172.21.15.91:6831","nonce":1}]},"public_addr":"172.21.15.91:6827/1","cluster_addr":"172.21.15.91:6829/1","heartbeat_back_addr":"172.21.15.91:6833/1","heartbeat_front_addr":"172.21.15.91:6831/1","state":["exists","up"]},{"osd":4,"uuid":"0c8c0189-2dd0-4a79-9e1f-8533d73b4480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6801","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1},{"type":"v1","addr":"172.21.15.107:6803","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1},{"type":"v1","addr":"172.21.15.107:6807","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1},{"type":"v1","addr":"172.21.15.107:6805","nonce":1}]},"public_addr":"172.21.15.107:6801/1","cluster_addr":"172.21.15.107:6803/1","heartbeat_back_addr":"172.21.15.107:6807/1","heartbeat_front_addr":"172.21.15.107:6805/1","state":["exists","up"]},{"osd":5,"uuid":"3a1924d5-2253-435f-a199-2286ef771c27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6809","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1},{"type":"v1","addr":"172.21.15.107:6811","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1},{"type":"v1","addr":"172.21.15.107:6815","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1},{"type":"v1","addr":"172.21.15.107:6813","nonce":1}]},"public_addr":"172.21.15.107:6809/1","cluster_addr":"172.21.15.107:6811/1","heartbeat_back_addr":"172.21.15.107:6815/1","heartbeat_front_addr":"172.21.15.107:6813/1","state":["exists","up"]},{"osd":6,"uuid":"56e1405c-eeda-4e92-b8de-b330fe2b9a7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6817","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1},{"type":"v1","addr":"172.21.15.107:6819","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1},{"type":"v1","addr":"172.21.15.107:6823","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1},{"type":"v1","addr":"172.21.15.107:6821","nonce":1}]},"public_addr":"172.21.15.107:6817/1","cluster_addr":"172.21.15.107:6819/1","heartbeat_back_addr":"172.21.15.107:6823/1","heartbeat_front_addr":"172.21.15.107:6821/1","state":["exists","up"]},{"osd":7,"uuid":"9a55297d-6a61-444a-a3b1-66293cb3dfe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6825","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1},{"type":"v1","addr":"172.21.15.107:6827","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1},{"type":"v1","addr":"172.21.15.107:6831","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1},{"type":"v1","addr":"172.21.15.107:6829","nonce":1}]},"public_addr":"172.21.15.107:6825/1","cluster_addr":"172.21.15.107:6827/1","heartbeat_back_addr":"172.21.15.107:6831/1","heartbeat_front_addr":"172.21.15.107:6829/1","state":["exists","up"]},{"osd":8,"uuid":"71cd6cef-0dd6-4744-bd1e-d407cc9f4e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6801","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1},{"type":"v1","addr":"172.21.15.43:6803","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1},{"type":"v1","addr":"172.21.15.43:6807","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1},{"type":"v1","addr":"172.21.15.43:6805","nonce":1}]},"public_addr":"172.21.15.43:6801/1","cluster_addr":"172.21.15.43:6803/1","heartbeat_back_addr":"172.21.15.43:6807/1","heartbeat_front_addr":"172.21.15.43:6805/1","state":["exists","up"]},{"osd":9,"uuid":"7c1f2ce9-60f1-49ce-8bdb-1608d4a5b47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6809","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1},{"type":"v1","addr":"172.21.15.43:6811","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1},{"type":"v1","addr":"172.21.15.43:6815","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1},{"type":"v1","addr":"172.21.15.43:6813","nonce":1}]},"public_addr":"172.21.15.43:6809/1","cluster_addr":"172.21.15.43:6811/1","heartbeat_back_addr":"172.21.15.43:6815/1","heartbeat_front_addr":"172.21.15.43:6813/1","state":["exists","up"]},{"osd":10,"uuid":"81aebf87-7a3a-4dc2-af79-2f21da7a7832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6817","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1},{"type":"v1","addr":"172.21.15.43:6819","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1},{"type":"v1","addr":"172.21.15.43:6823","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1},{"type":"v1","addr":"172.21.15.43:6821","nonce":1}]},"public_addr":"172.21.15.43:6817/1","cluster_addr":"172.21.15.43:6819/1","heartbeat_back_addr":"172.21.15.43:6823/1","heartbeat_front_addr":"172.21.15.43:6821/1","state":["exists","up"]},{"osd":11,"uuid":"d025b7a0-a2cb-4769-9908-5d1a93507c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6825","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1},{"type":"v1","addr":"172.21.15.43:6827","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1},{"type":"v1","addr":"172.21.15.43:6831","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1},{"type":"v1","addr":"172.21.15.43:6829","nonce":1}]},"public_addr":"172.21.15.43:6825/1","cluster_addr":"172.21.15.43:6827/1","heartbeat_back_addr":"172.21.15.43:6831/1","heartbeat_front_addr":"172.21.15.43:6829/1","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:15.319727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:36.199506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:55.229369+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:14.187705+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:34.231175+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:55.017979+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:15.435592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:35.215727+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:54.884875+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:14.037308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:34.562350+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:54.850928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.91:6801/1":"2019-12-23T22:20:16.666965+0000","172.21.15.91:6800/1":"2019-12-23T22:20:16.666965+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-22T22:27:01.087 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2019-12-22T22:27:01.087 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd dump --format=json 2019-12-22T22:27:03.293 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:27:03.294 INFO:teuthology.orchestra.run.smithi091.stdout:{"epoch":63,"fsid":"147630da-2509-11ea-8284-001a4aab830c","created":"2019-12-22T22:20:00.154953+0000","modified":"2019-12-22T22:26:56.979964+0000","last_up_change":"2019-12-22T22:26:55.970471+0000","last_in_change":"2019-12-22T22:26:55.970471+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":25,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":12,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"4520fba2-2eb9-4a60-8945-d2d3a0e5128d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.91:6803","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6804","nonce":1},{"type":"v1","addr":"172.21.15.91:6805","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6808","nonce":1},{"type":"v1","addr":"172.21.15.91:6809","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6806","nonce":1},{"type":"v1","addr":"172.21.15.91:6807","nonce":1}]},"public_addr":"172.21.15.91:6803/1","cluster_addr":"172.21.15.91:6805/1","heartbeat_back_addr":"172.21.15.91:6809/1","heartbeat_front_addr":"172.21.15.91:6807/1","state":["exists","up"]},{"osd":1,"uuid":"ff7b9947-09d3-4363-9073-f074112a0eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.91:6811","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6812","nonce":1},{"type":"v1","addr":"172.21.15.91:6813","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6816","nonce":1},{"type":"v1","addr":"172.21.15.91:6817","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6814","nonce":1},{"type":"v1","addr":"172.21.15.91:6815","nonce":1}]},"public_addr":"172.21.15.91:6811/1","cluster_addr":"172.21.15.91:6813/1","heartbeat_back_addr":"172.21.15.91:6817/1","heartbeat_front_addr":"172.21.15.91:6815/1","state":["exists","up"]},{"osd":2,"uuid":"ccdb35fb-e575-4f63-8f4a-3dc721d55812","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.91:6819","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6820","nonce":1},{"type":"v1","addr":"172.21.15.91:6821","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6824","nonce":1},{"type":"v1","addr":"172.21.15.91:6825","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6822","nonce":1},{"type":"v1","addr":"172.21.15.91:6823","nonce":1}]},"public_addr":"172.21.15.91:6819/1","cluster_addr":"172.21.15.91:6821/1","heartbeat_back_addr":"172.21.15.91:6825/1","heartbeat_front_addr":"172.21.15.91:6823/1","state":["exists","up"]},{"osd":3,"uuid":"5eac4685-b5d5-41e5-84d7-11702f96d78e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.91:6827","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6828","nonce":1},{"type":"v1","addr":"172.21.15.91:6829","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6832","nonce":1},{"type":"v1","addr":"172.21.15.91:6833","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.91:6830","nonce":1},{"type":"v1","addr":"172.21.15.91:6831","nonce":1}]},"public_addr":"172.21.15.91:6827/1","cluster_addr":"172.21.15.91:6829/1","heartbeat_back_addr":"172.21.15.91:6833/1","heartbeat_front_addr":"172.21.15.91:6831/1","state":["exists","up"]},{"osd":4,"uuid":"0c8c0189-2dd0-4a79-9e1f-8533d73b4480","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6801","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1},{"type":"v1","addr":"172.21.15.107:6803","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1},{"type":"v1","addr":"172.21.15.107:6807","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1},{"type":"v1","addr":"172.21.15.107:6805","nonce":1}]},"public_addr":"172.21.15.107:6801/1","cluster_addr":"172.21.15.107:6803/1","heartbeat_back_addr":"172.21.15.107:6807/1","heartbeat_front_addr":"172.21.15.107:6805/1","state":["exists","up"]},{"osd":5,"uuid":"3a1924d5-2253-435f-a199-2286ef771c27","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6809","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":1},{"type":"v1","addr":"172.21.15.107:6811","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":1},{"type":"v1","addr":"172.21.15.107:6815","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":1},{"type":"v1","addr":"172.21.15.107:6813","nonce":1}]},"public_addr":"172.21.15.107:6809/1","cluster_addr":"172.21.15.107:6811/1","heartbeat_back_addr":"172.21.15.107:6815/1","heartbeat_front_addr":"172.21.15.107:6813/1","state":["exists","up"]},{"osd":6,"uuid":"56e1405c-eeda-4e92-b8de-b330fe2b9a7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6817","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1},{"type":"v1","addr":"172.21.15.107:6819","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1},{"type":"v1","addr":"172.21.15.107:6823","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1},{"type":"v1","addr":"172.21.15.107:6821","nonce":1}]},"public_addr":"172.21.15.107:6817/1","cluster_addr":"172.21.15.107:6819/1","heartbeat_back_addr":"172.21.15.107:6823/1","heartbeat_front_addr":"172.21.15.107:6821/1","state":["exists","up"]},{"osd":7,"uuid":"9a55297d-6a61-444a-a3b1-66293cb3dfe9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.107:6825","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1},{"type":"v1","addr":"172.21.15.107:6827","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1},{"type":"v1","addr":"172.21.15.107:6831","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1},{"type":"v1","addr":"172.21.15.107:6829","nonce":1}]},"public_addr":"172.21.15.107:6825/1","cluster_addr":"172.21.15.107:6827/1","heartbeat_back_addr":"172.21.15.107:6831/1","heartbeat_front_addr":"172.21.15.107:6829/1","state":["exists","up"]},{"osd":8,"uuid":"71cd6cef-0dd6-4744-bd1e-d407cc9f4e11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6801","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1},{"type":"v1","addr":"172.21.15.43:6803","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1},{"type":"v1","addr":"172.21.15.43:6807","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1},{"type":"v1","addr":"172.21.15.43:6805","nonce":1}]},"public_addr":"172.21.15.43:6801/1","cluster_addr":"172.21.15.43:6803/1","heartbeat_back_addr":"172.21.15.43:6807/1","heartbeat_front_addr":"172.21.15.43:6805/1","state":["exists","up"]},{"osd":9,"uuid":"7c1f2ce9-60f1-49ce-8bdb-1608d4a5b47a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6809","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1},{"type":"v1","addr":"172.21.15.43:6811","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1},{"type":"v1","addr":"172.21.15.43:6815","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1},{"type":"v1","addr":"172.21.15.43:6813","nonce":1}]},"public_addr":"172.21.15.43:6809/1","cluster_addr":"172.21.15.43:6811/1","heartbeat_back_addr":"172.21.15.43:6815/1","heartbeat_front_addr":"172.21.15.43:6813/1","state":["exists","up"]},{"osd":10,"uuid":"81aebf87-7a3a-4dc2-af79-2f21da7a7832","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6817","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":1},{"type":"v1","addr":"172.21.15.43:6819","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":1},{"type":"v1","addr":"172.21.15.43:6823","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":1},{"type":"v1","addr":"172.21.15.43:6821","nonce":1}]},"public_addr":"172.21.15.43:6817/1","cluster_addr":"172.21.15.43:6819/1","heartbeat_back_addr":"172.21.15.43:6823/1","heartbeat_front_addr":"172.21.15.43:6821/1","state":["exists","up"]},{"osd":11,"uuid":"d025b7a0-a2cb-4769-9908-5d1a93507c1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6825","nonce":1}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":1},{"type":"v1","addr":"172.21.15.43:6827","nonce":1}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":1},{"type":"v1","addr":"172.21.15.43:6831","nonce":1}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":1},{"type":"v1","addr":"172.21.15.43:6829","nonce":1}]},"public_addr":"172.21.15.43:6825/1","cluster_addr":"172.21.15.43:6827/1","heartbeat_back_addr":"172.21.15.43:6831/1","heartbeat_front_addr":"172.21.15.43:6829/1","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:15.319727+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:36.199506+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:55.229369+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:14.187705+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:34.231175+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:55.017979+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:15.435592+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:35.215727+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:54.884875+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:14.037308+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:34.562350+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-22T22:26:54.850928+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.15.91:6801/1":"2019-12-23T22:20:16.666965+0000","172.21.15.91:6800/1":"2019-12-23T22:20:16.666965+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-22T22:27:03.816 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2019-12-22T22:27:06.426 INFO:teuthology.orchestra.run.smithi091.stdout:50 2019-12-22T22:27:06.426 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2019-12-22T22:27:08.993 INFO:teuthology.orchestra.run.smithi091.stdout:45 2019-12-22T22:27:08.993 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2019-12-22T22:27:11.561 INFO:teuthology.orchestra.run.smithi091.stdout:43 2019-12-22T22:27:11.561 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2019-12-22T22:27:14.180 INFO:teuthology.orchestra.run.smithi091.stdout:39 2019-12-22T22:27:14.180 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2019-12-22T22:27:16.810 INFO:teuthology.orchestra.run.smithi091.stdout:36 2019-12-22T22:27:16.811 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2019-12-22T22:27:19.570 INFO:teuthology.orchestra.run.smithi091.stdout:32 2019-12-22T22:27:19.571 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2019-12-22T22:27:22.261 INFO:teuthology.orchestra.run.smithi091.stdout:28 2019-12-22T22:27:22.262 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2019-12-22T22:27:24.862 INFO:teuthology.orchestra.run.smithi091.stdout:23 2019-12-22T22:27:24.862 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2019-12-22T22:27:27.428 INFO:teuthology.orchestra.run.smithi091.stdout:20 2019-12-22T22:27:27.429 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2019-12-22T22:27:29.978 INFO:teuthology.orchestra.run.smithi091.stdout:17 2019-12-22T22:27:29.978 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2019-12-22T22:27:32.492 INFO:teuthology.orchestra.run.smithi091.stdout:13 2019-12-22T22:27:32.492 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2019-12-22T22:27:35.026 INFO:teuthology.orchestra.run.smithi091.stdout:10 2019-12-22T22:27:35.027 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd last-stat-seq osd.0 2019-12-22T22:27:37.152 INFO:teuthology.orchestra.run.smithi091.stdout:55 2019-12-22T22:27:37.657 INFO:tasks.cephadm.ceph_manager.ceph:need seq 50 got 55 for osd.0 2019-12-22T22:27:37.658 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd last-stat-seq osd.1 2019-12-22T22:27:39.856 INFO:teuthology.orchestra.run.smithi091.stdout:51 2019-12-22T22:27:40.333 INFO:tasks.cephadm.ceph_manager.ceph:need seq 45 got 51 for osd.1 2019-12-22T22:27:40.333 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd last-stat-seq osd.2 2019-12-22T22:27:42.491 INFO:teuthology.orchestra.run.smithi091.stdout:49 2019-12-22T22:27:43.008 INFO:tasks.cephadm.ceph_manager.ceph:need seq 43 got 49 for osd.2 2019-12-22T22:27:43.008 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd last-stat-seq osd.3 2019-12-22T22:27:45.153 INFO:teuthology.orchestra.run.smithi091.stdout:46 2019-12-22T22:27:45.647 INFO:tasks.cephadm.ceph_manager.ceph:need seq 39 got 46 for osd.3 2019-12-22T22:27:45.648 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd last-stat-seq osd.4 2019-12-22T22:27:47.762 INFO:teuthology.orchestra.run.smithi091.stdout:42 2019-12-22T22:27:48.224 INFO:tasks.cephadm.ceph_manager.ceph:need seq 36 got 42 for osd.4 2019-12-22T22:27:48.225 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd last-stat-seq osd.5 2019-12-22T22:27:50.460 INFO:teuthology.orchestra.run.smithi091.stdout:38 2019-12-22T22:27:50.973 INFO:tasks.cephadm.ceph_manager.ceph:need seq 32 got 38 for osd.5 2019-12-22T22:27:50.973 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd last-stat-seq osd.6 2019-12-22T22:27:53.231 INFO:teuthology.orchestra.run.smithi091.stdout:35 2019-12-22T22:27:53.724 INFO:tasks.cephadm.ceph_manager.ceph:need seq 28 got 35 for osd.6 2019-12-22T22:27:53.725 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd last-stat-seq osd.7 2019-12-22T22:27:55.895 INFO:teuthology.orchestra.run.smithi091.stdout:29 2019-12-22T22:27:56.421 INFO:tasks.cephadm.ceph_manager.ceph:need seq 23 got 29 for osd.7 2019-12-22T22:27:56.422 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd last-stat-seq osd.8 2019-12-22T22:27:58.665 INFO:teuthology.orchestra.run.smithi091.stdout:26 2019-12-22T22:27:59.163 INFO:tasks.cephadm.ceph_manager.ceph:need seq 20 got 26 for osd.8 2019-12-22T22:27:59.164 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd last-stat-seq osd.9 2019-12-22T22:28:01.337 INFO:teuthology.orchestra.run.smithi091.stdout:23 2019-12-22T22:28:01.846 INFO:tasks.cephadm.ceph_manager.ceph:need seq 17 got 23 for osd.9 2019-12-22T22:28:01.846 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd last-stat-seq osd.10 2019-12-22T22:28:04.003 INFO:teuthology.orchestra.run.smithi091.stdout:19 2019-12-22T22:28:04.519 INFO:tasks.cephadm.ceph_manager.ceph:need seq 13 got 19 for osd.10 2019-12-22T22:28:04.519 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd last-stat-seq osd.11 2019-12-22T22:28:06.761 INFO:teuthology.orchestra.run.smithi091.stdout:16 2019-12-22T22:28:07.289 INFO:tasks.cephadm.ceph_manager.ceph:need seq 10 got 16 for osd.11 2019-12-22T22:28:07.289 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2019-12-22T22:28:07.290 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph pg dump --format=json 2019-12-22T22:28:09.488 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:28:09.489 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2019-12-22T22:28:10.013 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2019-12-22T22:28:08.915636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":12,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1124941824,"kb_used":12590400,"kb_used_data":7392,"kb_used_omap":0,"kb_used_meta":12582912,"kb_avail":1112351424,"statfs":{"total":1151940427776,"available":1139047858176,"internally_reserved":12884901888,"allocated":7569408,"data_stored":3466248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12884901888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[],"pool_stats":[],"osd_stats":[{"osd":11,"up_from":0,"seq":17,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.618,"5min":0.618,"15min":0.618},"max":{"1min":2.561,"5min":2.561,"15min":2.561},"last":1.189},{"interface":"front","average":{"1min":0.962,"5min":0.962,"15min":0.962},"min":{"1min":0.641,"5min":0.641,"15min":0.641},"max":{"1min":1.942,"5min":1.942,"15min":1.942},"last":0.847}]},{"osd":1,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":0.856,"5min":0.856,"15min":0.856},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.791},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.707,"5min":0.707,"15min":0.707},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":0.997}]},{"osd":2,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.037,"5min":1.037,"15min":1.037},"min":{"1min":0.812,"5min":0.812,"15min":0.812},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.037},{"interface":"front","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.656,"5min":0.656,"15min":0.656},"max":{"1min":1.312,"5min":1.312,"15min":1.312},"last":0.762}]},{"osd":3,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.687,"5min":0.687,"15min":0.687},"max":{"1min":1.666,"5min":1.666,"15min":1.666},"last":0.728},{"interface":"front","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.692,"5min":0.692,"15min":0.692},"max":{"1min":2.688,"5min":2.688,"15min":2.688},"last":0.907}]},{"osd":4,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.533,"5min":1.533,"15min":1.533},"min":{"1min":0.955,"5min":0.955,"15min":0.955},"max":{"1min":3.629,"5min":3.629,"15min":3.629},"last":1.397},{"interface":"front","average":{"1min":1.270,"5min":1.270,"15min":1.270},"min":{"1min":0.761,"5min":0.761,"15min":0.761},"max":{"1min":1.894,"5min":1.894,"15min":1.894},"last":1.133}]},{"osd":5,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.820,"5min":0.820,"15min":0.820},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.108},{"interface":"front","average":{"1min":1.686,"5min":1.686,"15min":1.686},"min":{"1min":1.128,"5min":1.128,"15min":1.128},"max":{"1min":3.739,"5min":3.739,"15min":3.739},"last":1.300}]},{"osd":6,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":35.379,"5min":35.379,"15min":35.379},"min":{"1min":0.984,"5min":0.984,"15min":0.984},"max":{"1min":646.716,"5min":646.716,"15min":646.716},"last":1.340},{"interface":"front","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.905,"5min":0.905,"15min":0.905},"max":{"1min":2.199,"5min":2.199,"15min":2.199},"last":0.968}]},{"osd":7,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.558,"5min":1.558,"15min":1.558},"min":{"1min":1.116,"5min":1.116,"15min":1.116},"max":{"1min":2.283,"5min":2.283,"15min":2.283},"last":1.442},{"interface":"front","average":{"1min":1.566,"5min":1.566,"15min":1.566},"min":{"1min":1.204,"5min":1.204,"15min":1.204},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":1.257}]},{"osd":8,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.822,"5min":1.822,"15min":1.822},"min":{"1min":1.237,"5min":1.237,"15min":1.237},"max":{"1min":3.936,"5min":3.936,"15min":3.936},"last":1.600},{"interface":"front","average":{"1min":34.440,"5min":34.440,"15min":34.440},"min":{"1min":1.332,"5min":1.332,"15min":1.332},"max":{"1min":624.904,"5min":624.904,"15min":624.904},"last":1.633}]},{"osd":10,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.871,"5min":1.871,"15min":1.871},"min":{"1min":1.205,"5min":1.205,"15min":1.205},"max":{"1min":4.041,"5min":4.041,"15min":4.041},"last":1.494},{"interface":"front","average":{"1min":1.772,"5min":1.772,"15min":1.772},"min":{"1min":1.354,"5min":1.354,"15min":1.354},"max":{"1min":3.823,"5min":3.823,"15min":3.823},"last":1.546}]}]},{"osd":10,"up_from":0,"seq":20,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":1.404}]},{"osd":1,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.667,"5min":0.667,"15min":0.667},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.619},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.712,"5min":0.712,"15min":0.712},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":1.832}]},{"osd":2,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.549,"5min":0.549,"15min":0.549},"max":{"1min":1.685,"5min":1.685,"15min":1.685},"last":1.343},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.604,"5min":0.604,"15min":0.604},"max":{"1min":2.191,"5min":2.191,"15min":2.191},"last":1.792}]},{"osd":3,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.010,"5min":1.010,"15min":1.010},"min":{"1min":0.635,"5min":0.635,"15min":0.635},"max":{"1min":1.438,"5min":1.438,"15min":1.438},"last":1.657},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.883,"5min":0.883,"15min":0.883},"max":{"1min":1.646,"5min":1.646,"15min":1.646},"last":1.484}]},{"osd":4,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.588,"5min":1.588,"15min":1.588},"min":{"1min":1.003,"5min":1.003,"15min":1.003},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":1.423},{"interface":"front","average":{"1min":1.915,"5min":1.915,"15min":1.915},"min":{"1min":0.740,"5min":0.740,"15min":0.740},"max":{"1min":11.454,"5min":11.454,"15min":11.454},"last":1.013}]},{"osd":5,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.430,"5min":1.430,"15min":1.430},"min":{"1min":0.757,"5min":0.757,"15min":0.757},"max":{"1min":2.450,"5min":2.450,"15min":2.450},"last":1.573},{"interface":"front","average":{"1min":1.681,"5min":1.681,"15min":1.681},"min":{"1min":1.204,"5min":1.204,"15min":1.204},"max":{"1min":2.777,"5min":2.777,"15min":2.777},"last":1.050}]},{"osd":6,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":2.012,"5min":2.012,"15min":2.012},"min":{"1min":1.031,"5min":1.031,"15min":1.031},"max":{"1min":11.404,"5min":11.404,"15min":11.404},"last":0.857},{"interface":"front","average":{"1min":1.559,"5min":1.559,"15min":1.559},"min":{"1min":1.108,"5min":1.108,"15min":1.108},"max":{"1min":2.462,"5min":2.462,"15min":2.462},"last":0.824}]},{"osd":7,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":102.420,"5min":102.420,"15min":102.420},"min":{"1min":1.064,"5min":1.064,"15min":1.064},"max":{"1min":938.067,"5min":938.067,"15min":938.067},"last":1.242},{"interface":"front","average":{"1min":1.644,"5min":1.644,"15min":1.644},"min":{"1min":1.151,"5min":1.151,"15min":1.151},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.972}]},{"osd":8,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":2.393,"5min":2.393,"15min":2.393},"min":{"1min":1.068,"5min":1.068,"15min":1.068},"max":{"1min":11.238,"5min":11.238,"15min":11.238},"last":1.300},{"interface":"front","average":{"1min":2.286,"5min":2.286,"15min":2.286},"min":{"1min":0.928,"5min":0.928,"15min":0.928},"max":{"1min":11.482,"5min":11.482,"15min":11.482},"last":1.373}]},{"osd":9,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":2.448,"5min":2.448,"15min":2.448},"min":{"1min":1.242,"5min":1.242,"15min":1.242},"max":{"1min":11.328,"5min":11.328,"15min":11.328},"last":1.210},{"interface":"front","average":{"1min":2.308,"5min":2.308,"15min":2.308},"min":{"1min":1.182,"5min":1.182,"15min":1.182},"max":{"1min":11.191,"5min":11.191,"15min":11.191},"last":1.522}]},{"osd":11,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.723,"5min":1.723,"15min":1.723},"min":{"1min":1.306,"5min":1.306,"15min":1.306},"max":{"1min":2.336,"5min":2.336,"15min":2.336},"last":1.150},{"interface":"front","average":{"1min":1.915,"5min":1.915,"15min":1.915},"min":{"1min":1.594,"5min":1.594,"15min":1.594},"max":{"1min":2.620,"5min":2.620,"15min":2.620},"last":1.696}]}]},{"osd":9,"up_from":0,"seq":25,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.084,"5min":1.084,"15min":1.084},"min":{"1min":0.636,"5min":0.636,"15min":0.636},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.961},{"interface":"front","average":{"1min":0.973,"5min":0.973,"15min":0.973},"min":{"1min":0.604,"5min":0.604,"15min":0.604},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.840}]},{"osd":1,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.037,"5min":1.037,"15min":1.037},"min":{"1min":0.656,"5min":0.656,"15min":0.656},"max":{"1min":1.664,"5min":1.664,"15min":1.664},"last":0.659},{"interface":"front","average":{"1min":0.891,"5min":0.891,"15min":0.891},"min":{"1min":0.647,"5min":0.647,"15min":0.647},"max":{"1min":1.437,"5min":1.437,"15min":1.437},"last":0.749}]},{"osd":2,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.197,"5min":1.197,"15min":1.197},"min":{"1min":0.738,"5min":0.738,"15min":0.738},"max":{"1min":2.162,"5min":2.162,"15min":2.162},"last":1.144},{"interface":"front","average":{"1min":27.186,"5min":27.186,"15min":27.186},"min":{"1min":0.668,"5min":0.668,"15min":0.668},"max":{"1min":524.094,"5min":524.094,"15min":524.094},"last":1.050}]},{"osd":3,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":35.175,"5min":35.175,"15min":35.175},"min":{"1min":0.538,"5min":0.538,"15min":0.538},"max":{"1min":683.851,"5min":683.851,"15min":683.851},"last":1.101},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.599,"5min":0.599,"15min":0.599},"max":{"1min":2.067,"5min":2.067,"15min":2.067},"last":1.302}]},{"osd":4,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.219,"5min":1.219,"15min":1.219},"min":{"1min":0.840,"5min":0.840,"15min":0.840},"max":{"1min":1.574,"5min":1.574,"15min":1.574},"last":1.208},{"interface":"front","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.716,"5min":0.716,"15min":0.716},"max":{"1min":1.720,"5min":1.720,"15min":1.720},"last":1.250}]},{"osd":5,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.275,"5min":1.275,"15min":1.275},"min":{"1min":0.693,"5min":0.693,"15min":0.693},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.460},{"interface":"front","average":{"1min":1.533,"5min":1.533,"15min":1.533},"min":{"1min":1.070,"5min":1.070,"15min":1.070},"max":{"1min":2.252,"5min":2.252,"15min":2.252},"last":1.523}]},{"osd":6,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":14.726,"5min":14.726,"15min":14.726},"min":{"1min":1.131,"5min":1.131,"15min":1.131},"max":{"1min":264.274,"5min":264.274,"15min":264.274},"last":1.611},{"interface":"front","average":{"1min":1.342,"5min":1.342,"15min":1.342},"min":{"1min":1.016,"5min":1.016,"15min":1.016},"max":{"1min":1.660,"5min":1.660,"15min":1.660},"last":1.352}]},{"osd":7,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.453,"5min":1.453,"15min":1.453},"min":{"1min":0.757,"5min":0.757,"15min":0.757},"max":{"1min":1.965,"5min":1.965,"15min":1.965},"last":1.420},{"interface":"front","average":{"1min":1.697,"5min":1.697,"15min":1.697},"min":{"1min":1.248,"5min":1.248,"15min":1.248},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":1.573}]},{"osd":8,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.443,"5min":1.443,"15min":1.443},"min":{"1min":0.700,"5min":0.700,"15min":0.700},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.495},{"interface":"front","average":{"1min":1.567,"5min":1.567,"15min":1.567},"min":{"1min":0.919,"5min":0.919,"15min":0.919},"max":{"1min":2.023,"5min":2.023,"15min":2.023},"last":1.662}]},{"osd":10,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.718,"5min":1.718,"15min":1.718},"min":{"1min":1.381,"5min":1.381,"15min":1.381},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":1.392},{"interface":"front","average":{"1min":1.802,"5min":1.802,"15min":1.802},"min":{"1min":1.407,"5min":1.407,"15min":1.407},"max":{"1min":2.188,"5min":2.188,"15min":2.188},"last":1.724}]}]},{"osd":8,"up_from":0,"seq":29,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.122,"5min":1.090,"15min":1.084},"min":{"1min":0.631,"5min":0.551,"15min":0.551},"max":{"1min":4.025,"5min":4.025,"15min":4.025},"last":1.894},{"interface":"front","average":{"1min":1.276,"5min":25.673,"15min":29.739},"min":{"1min":0.624,"5min":0.507,"15min":0.507},"max":{"1min":3.963,"5min":586.314,"15min":586.314},"last":1.834}]},{"osd":1,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":16.711,"5min":4.286,"15min":2.215},"min":{"1min":0.688,"5min":0.593,"15min":0.593},"max":{"1min":312.656,"5min":312.656,"15min":312.656},"last":1.717},{"interface":"front","average":{"1min":1.201,"5min":1.077,"15min":1.057},"min":{"1min":0.674,"5min":0.589,"15min":0.589},"max":{"1min":4.160,"5min":4.160,"15min":4.160},"last":1.649}]},{"osd":2,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.289,"5min":1.222,"15min":1.211},"min":{"1min":0.513,"5min":0.513,"15min":0.513},"max":{"1min":3.922,"5min":3.922,"15min":3.922},"last":2.009},{"interface":"front","average":{"1min":1.393,"5min":1.299,"15min":1.283},"min":{"1min":0.873,"5min":0.653,"15min":0.653},"max":{"1min":4.084,"5min":4.084,"15min":4.084},"last":1.948}]},{"osd":3,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.304,"15min":1.310},"min":{"1min":0.582,"5min":0.582,"15min":0.582},"max":{"1min":4.349,"5min":4.349,"15min":4.349},"last":1.488},{"interface":"front","average":{"1min":1.022,"5min":29.273,"15min":33.981},"min":{"1min":0.579,"5min":0.579,"15min":0.579},"max":{"1min":4.267,"5min":667.228,"15min":667.228},"last":1.778}]},{"osd":4,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.132,"5min":1.352,"15min":1.388},"min":{"1min":0.790,"5min":0.717,"15min":0.717},"max":{"1min":1.520,"5min":2.569,"15min":2.569},"last":1.372},{"interface":"front","average":{"1min":1.449,"5min":16.140,"15min":18.588},"min":{"1min":1.056,"5min":0.805,"15min":0.805},"max":{"1min":1.838,"5min":365.920,"15min":365.920},"last":1.320}]},{"osd":5,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.480,"15min":1.524},"min":{"1min":0.883,"5min":0.883,"15min":0.883},"max":{"1min":1.784,"5min":2.683,"15min":2.683},"last":1.010},{"interface":"front","average":{"1min":19.072,"5min":5.025,"15min":2.684},"min":{"1min":0.683,"5min":0.683,"15min":0.683},"max":{"1min":356.189,"5min":356.189,"15min":356.189},"last":0.954}]},{"osd":6,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.630,"15min":1.646},"min":{"1min":1.289,"5min":0.945,"15min":0.945},"max":{"1min":1.926,"5min":2.406,"15min":2.406},"last":1.147},{"interface":"front","average":{"1min":1.608,"5min":1.629,"15min":1.633},"min":{"1min":0.830,"5min":0.830,"15min":0.830},"max":{"1min":1.960,"5min":2.715,"15min":2.715},"last":1.279}]},{"osd":7,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":12.654,"15min":14.559},"min":{"1min":0.842,"5min":0.570,"15min":0.570},"max":{"1min":1.623,"5min":282.817,"15min":282.817},"last":1.211},{"interface":"front","average":{"1min":1.645,"5min":1.688,"15min":1.695},"min":{"1min":1.338,"5min":0.806,"15min":0.806},"max":{"1min":1.982,"5min":2.525,"15min":2.525},"last":1.419}]},{"osd":9,"last update":"Sun Dec 22 22:27:16 2019","interfaces":[{"interface":"back","average":{"1min":52.511,"5min":52.511,"15min":52.511},"min":{"1min":0.525,"5min":0.525,"15min":0.525},"max":{"1min":686.274,"5min":686.274,"15min":686.274},"last":0.701},{"interface":"front","average":{"1min":1.675,"5min":1.675,"15min":1.675},"min":{"1min":0.464,"5min":0.464,"15min":0.464},"max":{"1min":2.587,"5min":2.587,"15min":2.587},"last":1.089}]},{"osd":10,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":19.663,"5min":19.663,"15min":19.663},"min":{"1min":0.732,"5min":0.732,"15min":0.732},"max":{"1min":358.731,"5min":358.731,"15min":358.731},"last":1.734},{"interface":"front","average":{"1min":1.829,"5min":1.829,"15min":1.829},"min":{"1min":0.843,"5min":0.843,"15min":0.843},"max":{"1min":2.791,"5min":2.791,"15min":2.791},"last":1.547}]}]},{"osd":7,"up_from":0,"seq":32,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":0.973,"5min":0.854,"15min":0.834},"min":{"1min":0.558,"5min":0.558,"15min":0.558},"max":{"1min":1.443,"5min":1.443,"15min":1.443},"last":1.584},{"interface":"front","average":{"1min":14.982,"5min":3.818,"15min":1.958},"min":{"1min":0.731,"5min":0.684,"15min":0.684},"max":{"1min":291.908,"5min":291.908,"15min":291.908},"last":0.915}]},{"osd":1,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.197,"15min":1.201},"min":{"1min":0.819,"5min":0.758,"15min":0.758},"max":{"1min":1.616,"5min":2.308,"15min":2.308},"last":0.804},{"interface":"front","average":{"1min":1.079,"5min":1.059,"15min":1.056},"min":{"1min":0.680,"5min":0.680,"15min":0.680},"max":{"1min":1.396,"5min":1.937,"15min":1.937},"last":1.017}]},{"osd":2,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":0.991,"5min":1.018,"15min":1.022},"min":{"1min":0.685,"5min":0.685,"15min":0.685},"max":{"1min":1.529,"5min":1.886,"15min":1.886},"last":1.118},{"interface":"front","average":{"1min":1.121,"5min":1.145,"15min":1.149},"min":{"1min":0.817,"5min":0.625,"15min":0.625},"max":{"1min":1.524,"5min":2.376,"15min":2.376},"last":0.864}]},{"osd":3,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":42.500,"15min":49.405},"min":{"1min":0.648,"5min":0.648,"15min":0.648},"max":{"1min":1.494,"5min":985.876,"15min":985.876},"last":1.664},{"interface":"front","average":{"1min":1.217,"5min":1.272,"15min":1.281},"min":{"1min":0.784,"5min":0.694,"15min":0.694},"max":{"1min":1.559,"5min":2.153,"15min":2.153},"last":1.074}]},{"osd":4,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.143,"15min":1.151},"min":{"1min":0.699,"5min":0.600,"15min":0.600},"max":{"1min":1.654,"5min":1.800,"15min":1.800},"last":0.968},{"interface":"front","average":{"1min":1.159,"5min":1.191,"15min":1.196},"min":{"1min":0.704,"5min":0.704,"15min":0.704},"max":{"1min":1.643,"5min":1.833,"15min":1.833},"last":1.446}]},{"osd":5,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.154,"5min":1.273,"15min":1.293},"min":{"1min":0.633,"5min":0.633,"15min":0.633},"max":{"1min":1.629,"5min":1.663,"15min":1.663},"last":1.321},{"interface":"front","average":{"1min":1.398,"5min":1.387,"15min":1.385},"min":{"1min":0.855,"5min":0.728,"15min":0.728},"max":{"1min":1.747,"5min":2.435,"15min":2.435},"last":1.260}]},{"osd":6,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.322,"5min":1.273,"15min":1.265},"min":{"1min":0.611,"5min":0.611,"15min":0.611},"max":{"1min":1.710,"5min":2.106,"15min":2.106},"last":1.364},{"interface":"front","average":{"1min":1.369,"5min":31.793,"15min":36.864},"min":{"1min":0.976,"5min":0.976,"15min":0.976},"max":{"1min":1.755,"5min":684.801,"15min":684.801},"last":1.540}]},{"osd":8,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":29.540,"5min":7.085,"15min":3.343},"min":{"1min":0.901,"5min":0.807,"15min":0.807},"max":{"1min":502.217,"5min":502.217,"15min":502.217},"last":1.745},{"interface":"front","average":{"1min":1.560,"5min":1.628,"15min":1.639},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":1.788,"5min":2.619,"15min":2.619},"last":1.501}]},{"osd":9,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.606,"5min":1.606,"15min":1.606},"min":{"1min":1.228,"5min":1.228,"15min":1.228},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.707},{"interface":"front","average":{"1min":46.020,"5min":46.020,"15min":46.020},"min":{"1min":1.173,"5min":1.173,"15min":1.173},"max":{"1min":891.130,"5min":891.130,"15min":891.130},"last":1.816}]},{"osd":10,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":1.671,"15min":1.671},"min":{"1min":1.193,"5min":1.193,"15min":1.193},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.395},{"interface":"front","average":{"1min":53.065,"5min":53.065,"15min":53.065},"min":{"1min":1.327,"5min":1.327,"15min":1.327},"max":{"1min":748.773,"5min":748.773,"15min":748.773},"last":1.628}]}]},{"osd":6,"up_from":0,"seq":38,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":1.053,"15min":1.083},"min":{"1min":0.601,"5min":0.601,"15min":0.601},"max":{"1min":1.682,"5min":1.682,"15min":1.682},"last":1.172},{"interface":"front","average":{"1min":0.974,"5min":1.074,"15min":1.090},"min":{"1min":0.689,"5min":0.689,"15min":0.689},"max":{"1min":1.528,"5min":1.677,"15min":1.677},"last":0.900}]},{"osd":1,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.893,"15min":0.895},"min":{"1min":0.626,"5min":0.591,"15min":0.591},"max":{"1min":1.364,"5min":1.793,"15min":1.793},"last":1.214},{"interface":"front","average":{"1min":0.920,"5min":19.041,"15min":22.061},"min":{"1min":0.611,"5min":0.588,"15min":0.588},"max":{"1min":1.628,"5min":363.766,"15min":363.766},"last":1.048}]},{"osd":2,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":5.483,"5min":1.961,"15min":1.374},"min":{"1min":0.674,"5min":0.674,"15min":0.674},"max":{"1min":89.000,"5min":89.000,"15min":89.000},"last":0.839},{"interface":"front","average":{"1min":1.033,"5min":46.178,"15min":53.702},"min":{"1min":0.721,"5min":0.721,"15min":0.721},"max":{"1min":1.550,"5min":904.141,"15min":904.141},"last":0.734}]},{"osd":3,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":16.675,"15min":19.261},"min":{"1min":0.902,"5min":0.834,"15min":0.834},"max":{"1min":1.511,"5min":312.765,"15min":312.765},"last":1.005},{"interface":"front","average":{"1min":1.160,"5min":1.184,"15min":1.188},"min":{"1min":0.719,"5min":0.719,"15min":0.719},"max":{"1min":1.468,"5min":1.771,"15min":1.771},"last":1.287}]},{"osd":4,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":47.470,"5min":10.303,"15min":4.109},"min":{"1min":0.727,"5min":0.535,"15min":0.535},"max":{"1min":882.404,"5min":882.404,"15min":882.404},"last":0.797},{"interface":"front","average":{"1min":1.110,"5min":1.036,"15min":1.024},"min":{"1min":0.653,"5min":0.595,"15min":0.595},"max":{"1min":1.572,"5min":1.572,"15min":1.572},"last":1.093}]},{"osd":5,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.197,"5min":1.164,"15min":1.158},"min":{"1min":0.843,"5min":0.680,"15min":0.680},"max":{"1min":1.453,"5min":1.839,"15min":1.839},"last":1.260},{"interface":"front","average":{"1min":1.296,"5min":1.257,"15min":1.251},"min":{"1min":0.960,"5min":0.896,"15min":0.896},"max":{"1min":1.706,"5min":1.804,"15min":1.804},"last":1.326}]},{"osd":7,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.311,"15min":1.307},"min":{"1min":1.073,"5min":1.030,"15min":1.030},"max":{"1min":1.590,"5min":1.675,"15min":1.675},"last":1.423},{"interface":"front","average":{"1min":1.392,"5min":1.350,"15min":1.343},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.656,"5min":1.721,"15min":1.721},"last":1.450}]},{"osd":8,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":49.165,"15min":57.102},"min":{"1min":1.168,"5min":1.168,"15min":1.168},"max":{"1min":1.972,"5min":844.789,"15min":844.789},"last":1.554},{"interface":"front","average":{"1min":1.408,"5min":1.431,"15min":1.435},"min":{"1min":1.200,"5min":1.183,"15min":1.183},"max":{"1min":1.840,"5min":1.840,"15min":1.840},"last":1.373}]},{"osd":9,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.531,"5min":1.531,"15min":1.531},"min":{"1min":1.113,"5min":1.113,"15min":1.113},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":1.596},{"interface":"front","average":{"1min":1.485,"5min":1.485,"15min":1.485},"min":{"1min":1.043,"5min":1.043,"15min":1.043},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.502}]},{"osd":10,"last update":"Sun Dec 22 22:27:37 2019","interfaces":[{"interface":"back","average":{"1min":1.497,"5min":1.497,"15min":1.497},"min":{"1min":1.156,"5min":1.156,"15min":1.156},"max":{"1min":1.866,"5min":1.866,"15min":1.866},"last":1.530},{"interface":"front","average":{"1min":1.596,"5min":1.596,"15min":1.596},"min":{"1min":1.000,"5min":1.000,"15min":1.000},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.625}]}]},{"osd":1,"up_from":0,"seq":57,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":0.805,"5min":0.645,"15min":0.595},"min":{"1min":0.525,"5min":0.338,"15min":0.338},"max":{"1min":1.279,"5min":1.279,"15min":1.279},"last":0.879},{"interface":"front","average":{"1min":0.901,"5min":0.829,"15min":0.735},"min":{"1min":0.507,"5min":0.381,"15min":0.381},"max":{"1min":1.370,"5min":1.511,"15min":1.511},"last":1.488}]},{"osd":2,"last update":"Sun Dec 22 22:28:05 2019","interfaces":[{"interface":"back","average":{"1min":1.011,"5min":0.885,"15min":0.797},"min":{"1min":0.500,"5min":0.391,"15min":0.391},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.161},{"interface":"front","average":{"1min":0.869,"5min":3.282,"15min":1.544},"min":{"1min":0.475,"5min":0.436,"15min":0.436},"max":{"1min":1.471,"5min":254.847,"15min":254.847},"last":0.817}]},{"osd":3,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":46.363,"5min":18.287,"15min":6.627},"min":{"1min":0.651,"5min":0.551,"15min":0.551},"max":{"1min":952.051,"5min":952.051,"15min":952.051},"last":1.299},{"interface":"front","average":{"1min":1.186,"5min":17.623,"15min":14.386},"min":{"1min":0.620,"5min":0.620,"15min":0.620},"max":{"1min":1.684,"5min":903.487,"15min":903.487},"last":1.549}]},{"osd":4,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":1.010,"5min":0.878,"15min":0.847},"min":{"1min":0.707,"5min":0.601,"15min":0.601},"max":{"1min":1.424,"5min":1.424,"15min":1.424},"last":0.965},{"interface":"front","average":{"1min":1.338,"5min":1.193,"15min":1.117},"min":{"1min":0.946,"5min":0.732,"15min":0.732},"max":{"1min":1.685,"5min":1.941,"15min":1.941},"last":1.592}]},{"osd":5,"last update":"Sun Dec 22 22:28:02 2019","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":0.977,"15min":0.957},"min":{"1min":0.851,"5min":0.676,"15min":0.676},"max":{"1min":1.521,"5min":1.521,"15min":1.521},"last":1.091},{"interface":"front","average":{"1min":12.769,"5min":3.568,"15min":2.000},"min":{"1min":0.809,"5min":0.809,"15min":0.809},"max":{"1min":263.068,"5min":263.068,"15min":263.068},"last":1.669}]},{"osd":6,"last update":"Sun Dec 22 22:27:23 2019","interfaces":[{"interface":"back","average":{"1min":1.410,"5min":1.411,"15min":1.411},"min":{"1min":0.972,"5min":0.972,"15min":0.972},"max":{"1min":2.000,"5min":2.315,"15min":2.315},"last":1.636},{"interface":"front","average":{"1min":1.019,"5min":1.013,"15min":1.012},"min":{"1min":0.736,"5min":0.736,"15min":0.736},"max":{"1min":1.527,"5min":1.673,"15min":1.673},"last":1.207}]},{"osd":7,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":1.400,"5min":1.473,"15min":1.485},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.861,"5min":2.238,"15min":2.238},"last":1.877},{"interface":"front","average":{"1min":6.449,"5min":2.532,"15min":1.879},"min":{"1min":0.963,"5min":0.963,"15min":0.963},"max":{"1min":101.788,"5min":101.788,"15min":101.788},"last":1.852}]},{"osd":8,"last update":"Sun Dec 22 22:28:02 2019","interfaces":[{"interface":"back","average":{"1min":1.189,"5min":1.189,"15min":1.189},"min":{"1min":0.826,"5min":0.826,"15min":0.826},"max":{"1min":1.594,"5min":1.594,"15min":1.594},"last":1.255},{"interface":"front","average":{"1min":1.196,"5min":1.228,"15min":1.233},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":1.372}]},{"osd":9,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.589,"15min":1.589},"min":{"1min":1.121,"5min":1.121,"15min":1.121},"max":{"1min":2.231,"5min":2.231,"15min":2.231},"last":1.815},{"interface":"front","average":{"1min":15.510,"5min":15.510,"15min":15.510},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":300.557,"5min":300.557,"15min":300.557},"last":1.524}]},{"osd":10,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":46.030,"5min":46.030,"15min":46.030},"min":{"1min":1.021,"5min":1.021,"15min":1.021},"max":{"1min":896.161,"5min":896.161,"15min":896.161},"last":1.451},{"interface":"front","average":{"1min":1.528,"5min":1.528,"15min":1.528},"min":{"1min":1.144,"5min":1.144,"15min":1.144},"max":{"1min":1.902,"5min":1.902,"15min":1.902},"last":1.774}]}]},{"osd":0,"up_from":0,"seq":62,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":0.912,"5min":0.792,"15min":0.752},"min":{"1min":0.582,"5min":0.291,"15min":0.291},"max":{"1min":1.381,"5min":1.720,"15min":1.720},"last":0.804},{"interface":"front","average":{"1min":1.068,"5min":0.864,"15min":0.760},"min":{"1min":0.659,"5min":0.349,"15min":0.349},"max":{"1min":1.570,"5min":1.597,"15min":1.597},"last":0.856}]},{"osd":2,"last update":"Sun Dec 22 22:26:03 2019","interfaces":[{"interface":"back","average":{"1min":0.938,"5min":0.922,"15min":0.919},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.347,"5min":1.798,"15min":1.798},"last":1.188},{"interface":"front","average":{"1min":38.337,"5min":38.242,"15min":38.226},"min":{"1min":0.580,"5min":0.552,"15min":0.552},"max":{"1min":601.565,"5min":601.565,"15min":601.565},"last":0.645}]},{"osd":3,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.047,"5min":0.844,"15min":0.776},"min":{"1min":0.616,"5min":0.540,"15min":0.540},"max":{"1min":1.722,"5min":1.832,"15min":1.832},"last":0.511},{"interface":"front","average":{"1min":0.724,"5min":12.273,"15min":4.552},"min":{"1min":0.525,"5min":0.525,"15min":0.525},"max":{"1min":1.155,"5min":867.546,"15min":867.546},"last":0.452}]},{"osd":4,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":0.927,"5min":0.888,"15min":0.848},"min":{"1min":0.673,"5min":0.664,"15min":0.664},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":1.088},{"interface":"front","average":{"1min":1.313,"5min":1.149,"15min":1.077},"min":{"1min":0.945,"5min":0.850,"15min":0.850},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.935}]},{"osd":5,"last update":"Sun Dec 22 22:28:04 2019","interfaces":[{"interface":"back","average":{"1min":1.273,"5min":1.207,"15min":1.156},"min":{"1min":0.875,"5min":0.802,"15min":0.802},"max":{"1min":1.922,"5min":1.922,"15min":1.922},"last":1.025},{"interface":"front","average":{"1min":0.907,"5min":0.982,"15min":0.970},"min":{"1min":0.654,"5min":0.654,"15min":0.654},"max":{"1min":1.647,"5min":1.887,"15min":1.887},"last":1.224}]},{"osd":6,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.073,"15min":1.070},"min":{"1min":0.709,"5min":0.709,"15min":0.709},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.447},{"interface":"front","average":{"1min":1.480,"5min":26.547,"15min":30.725},"min":{"1min":0.923,"5min":0.923,"15min":0.923},"max":{"1min":2.246,"5min":504.570,"15min":504.570},"last":1.175}]},{"osd":7,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.195,"15min":1.199},"min":{"1min":0.863,"5min":0.863,"15min":0.863},"max":{"1min":1.616,"5min":1.616,"15min":1.616},"last":1.370},{"interface":"front","average":{"1min":1.488,"5min":26.395,"15min":30.546},"min":{"1min":0.975,"5min":0.975,"15min":0.975},"max":{"1min":1.885,"5min":530.298,"15min":530.298},"last":1.319}]},{"osd":8,"last update":"Sun Dec 22 22:28:04 2019","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.303,"15min":1.319},"min":{"1min":0.815,"5min":0.805,"15min":0.805},"max":{"1min":1.771,"5min":1.771,"15min":1.771},"last":1.518},{"interface":"front","average":{"1min":46.831,"5min":10.652,"15min":4.622},"min":{"1min":0.892,"5min":0.892,"15min":0.892},"max":{"1min":817.964,"5min":817.964,"15min":817.964},"last":1.686}]},{"osd":9,"last update":"Sun Dec 22 22:27:17 2019","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.649,"15min":1.649},"min":{"1min":1.280,"5min":1.280,"15min":1.280},"max":{"1min":2.180,"5min":2.180,"15min":2.180},"last":1.606},{"interface":"front","average":{"1min":1.264,"5min":1.264,"15min":1.264},"min":{"1min":0.969,"5min":0.969,"15min":0.969},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.472}]},{"osd":10,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.639,"15min":1.639},"min":{"1min":1.130,"5min":1.130,"15min":1.130},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":1.816},{"interface":"front","average":{"1min":1.644,"5min":1.644,"15min":1.644},"min":{"1min":1.072,"5min":1.072,"15min":1.072},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":1.756}]},{"osd":11,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.160,"5min":1.160,"15min":1.160},"min":{"1min":0.772,"5min":0.772,"15min":0.772},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.409},{"interface":"front","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":0.853,"5min":0.853,"15min":0.853},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.545}]}]},{"osd":2,"up_from":0,"seq":54,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:04 2019","interfaces":[{"interface":"back","average":{"1min":0.768,"5min":0.615,"15min":0.553},"min":{"1min":0.426,"5min":0.398,"15min":0.398},"max":{"1min":1.461,"5min":1.461,"15min":1.461},"last":0.739},{"interface":"front","average":{"1min":0.782,"5min":0.650,"15min":0.607},"min":{"1min":0.542,"5min":0.425,"15min":0.425},"max":{"1min":1.128,"5min":1.128,"15min":1.128},"last":0.994}]},{"osd":1,"last update":"Sun Dec 22 22:27:04 2019","interfaces":[{"interface":"back","average":{"1min":0.880,"5min":0.779,"15min":0.732},"min":{"1min":0.510,"5min":0.501,"15min":0.501},"max":{"1min":1.555,"5min":1.555,"15min":1.555},"last":1.076},{"interface":"front","average":{"1min":0.836,"5min":0.744,"15min":0.722},"min":{"1min":0.611,"5min":0.478,"15min":0.478},"max":{"1min":1.184,"5min":1.184,"15min":1.184},"last":1.406}]},{"osd":3,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.029,"5min":0.868,"15min":0.818},"min":{"1min":0.711,"5min":0.563,"15min":0.563},"max":{"1min":1.261,"5min":1.261,"15min":1.261},"last":1.243},{"interface":"front","average":{"1min":0.952,"5min":0.840,"15min":0.818},"min":{"1min":0.611,"5min":0.525,"15min":0.525},"max":{"1min":1.930,"5min":1.930,"15min":1.930},"last":0.776}]},{"osd":4,"last update":"Sun Dec 22 22:27:42 2019","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.019,"15min":0.995},"min":{"1min":0.594,"5min":0.594,"15min":0.594},"max":{"1min":1.947,"5min":1.947,"15min":1.947},"last":0.858},{"interface":"front","average":{"1min":1.112,"5min":1.104,"15min":1.094},"min":{"1min":0.690,"5min":0.668,"15min":0.668},"max":{"1min":1.588,"5min":1.588,"15min":1.588},"last":1.152}]},{"osd":5,"last update":"Sun Dec 22 22:27:04 2019","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.021,"15min":1.007},"min":{"1min":0.856,"5min":0.689,"15min":0.689},"max":{"1min":1.394,"5min":1.585,"15min":1.585},"last":1.338},{"interface":"front","average":{"1min":1.226,"5min":1.144,"15min":1.130},"min":{"1min":0.771,"5min":0.765,"15min":0.765},"max":{"1min":1.515,"5min":1.626,"15min":1.626},"last":1.373}]},{"osd":6,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.116,"5min":1.071,"15min":1.063},"min":{"1min":0.707,"5min":0.707,"15min":0.707},"max":{"1min":1.998,"5min":2.034,"15min":2.034},"last":1.047},{"interface":"front","average":{"1min":1.232,"5min":23.528,"15min":27.244},"min":{"1min":0.793,"5min":0.793,"15min":0.793},"max":{"1min":2.354,"5min":587.877,"15min":587.877},"last":1.191}]},{"osd":7,"last update":"Sun Dec 22 22:27:42 2019","interfaces":[{"interface":"back","average":{"1min":1.180,"5min":1.195,"15min":1.197},"min":{"1min":0.730,"5min":0.730,"15min":0.730},"max":{"1min":2.066,"5min":2.066,"15min":2.066},"last":0.936},{"interface":"front","average":{"1min":1.314,"5min":1.283,"15min":1.278},"min":{"1min":0.780,"5min":0.780,"15min":0.780},"max":{"1min":2.096,"5min":2.096,"15min":2.096},"last":1.286}]},{"osd":8,"last update":"Sun Dec 22 22:27:57 2019","interfaces":[{"interface":"back","average":{"1min":37.270,"5min":8.562,"15min":3.777},"min":{"1min":0.910,"5min":0.910,"15min":0.910},"max":{"1min":648.474,"5min":648.474,"15min":648.474},"last":1.493},{"interface":"front","average":{"1min":40.243,"5min":9.160,"15min":3.980},"min":{"1min":0.690,"5min":0.690,"15min":0.690},"max":{"1min":705.146,"5min":705.146,"15min":705.146},"last":1.117}]},{"osd":9,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":47.812,"5min":47.812,"15min":47.812},"min":{"1min":1.126,"5min":1.126,"15min":1.126},"max":{"1min":834.497,"5min":834.497,"15min":834.497},"last":1.428},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":1.017,"5min":1.017,"15min":1.017},"max":{"1min":1.749,"5min":1.749,"15min":1.749},"last":1.527}]},{"osd":10,"last update":"Sun Dec 22 22:27:42 2019","interfaces":[{"interface":"back","average":{"1min":59.261,"5min":59.261,"15min":59.261},"min":{"1min":1.117,"5min":1.117,"15min":1.117},"max":{"1min":984.644,"5min":984.644,"15min":984.644},"last":1.460},{"interface":"front","average":{"1min":1.477,"5min":1.477,"15min":1.477},"min":{"1min":0.897,"5min":0.897,"15min":0.897},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.565}]}]},{"osd":3,"up_from":0,"seq":51,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:26:18 2019","interfaces":[{"interface":"back","average":{"1min":0.828,"5min":35.683,"15min":41.492},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.248,"5min":961.866,"15min":961.866},"last":1.531},{"interface":"front","average":{"1min":0.810,"5min":0.701,"15min":0.683},"min":{"1min":0.510,"5min":0.510,"15min":0.510},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.753}]},{"osd":1,"last update":"Sun Dec 22 22:27:21 2019","interfaces":[{"interface":"back","average":{"1min":0.865,"5min":0.740,"15min":0.674},"min":{"1min":0.512,"5min":0.498,"15min":0.498},"max":{"1min":1.178,"5min":2.091,"15min":2.091},"last":0.529},{"interface":"front","average":{"1min":28.452,"5min":6.305,"15min":2.590},"min":{"1min":0.407,"5min":0.407,"15min":0.407},"max":{"1min":495.556,"5min":495.556,"15min":495.556},"last":0.618}]},{"osd":2,"last update":"Sun Dec 22 22:27:21 2019","interfaces":[{"interface":"back","average":{"1min":5.722,"5min":23.239,"15min":32.091},"min":{"1min":0.473,"5min":0.473,"15min":0.473},"max":{"1min":87.773,"5min":750.955,"15min":750.955},"last":0.475},{"interface":"front","average":{"1min":49.085,"5min":10.532,"15min":4.098},"min":{"1min":0.541,"5min":0.541,"15min":0.541},"max":{"1min":866.045,"5min":866.045,"15min":866.045},"last":0.864}]},{"osd":4,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.186,"15min":1.186},"min":{"1min":0.730,"5min":0.730,"15min":0.730},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.179},{"interface":"front","average":{"1min":0.900,"5min":1.007,"15min":1.023},"min":{"1min":0.663,"5min":0.663,"15min":0.663},"max":{"1min":1.231,"5min":1.657,"15min":1.657},"last":1.004}]},{"osd":5,"last update":"Sun Dec 22 22:28:00 2019","interfaces":[{"interface":"back","average":{"1min":39.628,"5min":8.791,"15min":3.613},"min":{"1min":0.574,"5min":0.574,"15min":0.574},"max":{"1min":501.574,"5min":501.574,"15min":501.574},"last":0.925},{"interface":"front","average":{"1min":1.165,"5min":1.205,"15min":1.201},"min":{"1min":0.678,"5min":0.678,"15min":0.678},"max":{"1min":1.649,"5min":1.845,"15min":1.845},"last":0.818}]},{"osd":6,"last update":"Sun Dec 22 22:27:21 2019","interfaces":[{"interface":"back","average":{"1min":28.463,"5min":6.643,"15min":3.007},"min":{"1min":0.650,"5min":0.650,"15min":0.650},"max":{"1min":246.895,"5min":246.895,"15min":246.895},"last":0.683},{"interface":"front","average":{"1min":11.539,"5min":3.329,"15min":1.961},"min":{"1min":0.597,"5min":0.597,"15min":0.597},"max":{"1min":184.604,"5min":184.604,"15min":184.604},"last":1.099}]},{"osd":7,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":14.350,"15min":16.570},"min":{"1min":0.802,"5min":0.768,"15min":0.768},"max":{"1min":1.493,"5min":347.131,"15min":347.131},"last":0.787},{"interface":"front","average":{"1min":1.230,"5min":1.334,"15min":1.352},"min":{"1min":0.775,"5min":0.775,"15min":0.775},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.977}]},{"osd":8,"last update":"Sun Dec 22 22:28:00 2019","interfaces":[{"interface":"back","average":{"1min":1.372,"5min":1.411,"15min":1.417},"min":{"1min":0.885,"5min":0.885,"15min":0.885},"max":{"1min":2.279,"5min":2.279,"15min":2.279},"last":1.074},{"interface":"front","average":{"1min":1.366,"5min":1.420,"15min":1.429},"min":{"1min":0.865,"5min":0.865,"15min":0.865},"max":{"1min":1.838,"5min":2.137,"15min":2.137},"last":1.315}]},{"osd":9,"last update":"Sun Dec 22 22:27:21 2019","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":1.001,"5min":1.001,"15min":1.001},"max":{"1min":1.658,"5min":1.658,"15min":1.658},"last":0.957},{"interface":"front","average":{"1min":1.489,"5min":1.489,"15min":1.489},"min":{"1min":1.019,"5min":1.019,"15min":1.019},"max":{"1min":2.025,"5min":2.025,"15min":2.025},"last":1.216}]},{"osd":10,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":6.989,"5min":6.989,"15min":6.989},"min":{"1min":0.978,"5min":0.978,"15min":0.978},"max":{"1min":110.434,"5min":110.434,"15min":110.434},"last":1.034},{"interface":"front","average":{"1min":1.545,"5min":1.545,"15min":1.545},"min":{"1min":0.928,"5min":0.928,"15min":0.928},"max":{"1min":7.829,"5min":7.829,"15min":7.829},"last":0.905}]},{"osd":11,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.441,"5min":1.441,"15min":1.441},"min":{"1min":0.952,"5min":0.952,"15min":0.952},"max":{"1min":2.313,"5min":2.313,"15min":2.313},"last":1.150},{"interface":"front","average":{"1min":1.445,"5min":1.445,"15min":1.445},"min":{"1min":1.052,"5min":1.052,"15min":1.052},"max":{"1min":1.781,"5min":1.781,"15min":1.781},"last":1.252}]}]},{"osd":4,"up_from":0,"seq":46,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":43.486,"5min":9.677,"15min":3.841},"min":{"1min":0.734,"5min":0.627,"15min":0.627},"max":{"1min":764.951,"5min":764.951,"15min":764.951},"last":0.876},{"interface":"front","average":{"1min":1.035,"5min":26.738,"15min":31.536},"min":{"1min":0.685,"5min":0.614,"15min":0.614},"max":{"1min":1.430,"5min":660.413,"15min":660.413},"last":1.242}]},{"osd":1,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":1.108,"5min":0.943,"15min":0.901},"min":{"1min":0.884,"5min":0.585,"15min":0.585},"max":{"1min":1.451,"5min":1.451,"15min":1.451},"last":1.007},{"interface":"front","average":{"1min":0.897,"5min":0.912,"15min":0.916},"min":{"1min":0.645,"5min":0.626,"15min":0.626},"max":{"1min":1.192,"5min":1.335,"15min":1.335},"last":0.901}]},{"osd":2,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":0.913,"5min":0.923,"15min":0.911},"min":{"1min":0.698,"5min":0.677,"15min":0.677},"max":{"1min":1.124,"5min":1.223,"15min":1.223},"last":0.689},{"interface":"front","average":{"1min":1.141,"5min":1.052,"15min":1.010},"min":{"1min":0.613,"5min":0.613,"15min":0.613},"max":{"1min":1.660,"5min":1.993,"15min":1.993},"last":0.756}]},{"osd":3,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":21.111,"5min":20.868,"15min":25.206},"min":{"1min":0.684,"5min":0.577,"15min":0.577},"max":{"1min":361.413,"5min":503.582,"15min":503.582},"last":1.068},{"interface":"front","average":{"1min":1.010,"5min":0.978,"15min":0.947},"min":{"1min":0.738,"5min":0.585,"15min":0.585},"max":{"1min":1.347,"5min":1.496,"15min":1.496},"last":1.162}]},{"osd":5,"last update":"Sun Dec 22 22:27:57 2019","interfaces":[{"interface":"back","average":{"1min":10.611,"5min":7.111,"15min":3.013},"min":{"1min":0.672,"5min":0.477,"15min":0.477},"max":{"1min":200.826,"5min":357.229,"15min":357.229},"last":1.289},{"interface":"front","average":{"1min":0.957,"5min":0.918,"15min":0.910},"min":{"1min":0.632,"5min":0.456,"15min":0.456},"max":{"1min":1.204,"5min":1.359,"15min":1.359},"last":1.101}]},{"osd":6,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":3.086,"15min":3.410},"min":{"1min":0.585,"5min":0.495,"15min":0.495},"max":{"1min":1.708,"5min":49.709,"15min":49.709},"last":1.324},{"interface":"front","average":{"1min":1.371,"5min":31.997,"15min":37.101},"min":{"1min":0.889,"5min":0.734,"15min":0.734},"max":{"1min":1.732,"5min":768.708,"15min":768.708},"last":1.359}]},{"osd":7,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.431,"15min":1.431},"min":{"1min":1.133,"5min":1.004,"15min":1.004},"max":{"1min":1.766,"5min":2.058,"15min":2.058},"last":1.202},{"interface":"front","average":{"1min":1.409,"5min":1.289,"15min":1.269},"min":{"1min":1.193,"5min":0.814,"15min":0.814},"max":{"1min":1.638,"5min":1.695,"15min":1.695},"last":1.466}]},{"osd":8,"last update":"Sun Dec 22 22:27:57 2019","interfaces":[{"interface":"back","average":{"1min":35.376,"5min":8.328,"15min":3.820},"min":{"1min":1.133,"5min":1.133,"15min":1.133},"max":{"1min":678.500,"5min":678.500,"15min":678.500},"last":1.403},{"interface":"front","average":{"1min":42.957,"5min":9.778,"15min":4.248},"min":{"1min":1.182,"5min":1.055,"15min":1.055},"max":{"1min":831.286,"5min":831.286,"15min":831.286},"last":1.539}]},{"osd":9,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.529,"5min":1.529,"15min":1.529},"min":{"1min":1.310,"5min":1.310,"15min":1.310},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":1.570},{"interface":"front","average":{"1min":1.480,"5min":1.480,"15min":1.480},"min":{"1min":1.227,"5min":1.227,"15min":1.227},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":1.605}]},{"osd":10,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":18.270,"5min":18.270,"15min":18.270},"min":{"1min":1.313,"5min":1.313,"15min":1.313},"max":{"1min":302.146,"5min":302.146,"15min":302.146},"last":1.501},{"interface":"front","average":{"1min":1.541,"5min":1.541,"15min":1.541},"min":{"1min":1.340,"5min":1.340,"15min":1.340},"max":{"1min":1.767,"5min":1.767,"15min":1.767},"last":1.675}]}]},{"osd":5,"up_from":0,"seq":42,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.189,"15min":1.258},"min":{"1min":0.611,"5min":0.557,"15min":0.557},"max":{"1min":2.727,"5min":7.646,"15min":7.646},"last":1.431},{"interface":"front","average":{"1min":1.145,"5min":22.091,"15min":31.454},"min":{"1min":0.743,"5min":0.548,"15min":0.548},"max":{"1min":2.694,"5min":803.966,"15min":803.966},"last":1.125}]},{"osd":1,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":12.150,"15min":17.025},"min":{"1min":0.679,"5min":0.553,"15min":0.553},"max":{"1min":8.832,"5min":456.064,"15min":456.064},"last":1.046},{"interface":"front","average":{"1min":1.067,"5min":1.152,"15min":1.219},"min":{"1min":0.632,"5min":0.619,"15min":0.619},"max":{"1min":1.543,"5min":7.728,"15min":7.728},"last":0.916}]},{"osd":2,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.406,"5min":8.846,"15min":12.250},"min":{"1min":0.658,"5min":0.651,"15min":0.651},"max":{"1min":8.927,"5min":319.127,"15min":319.127},"last":1.177},{"interface":"front","average":{"1min":1.141,"5min":1.268,"15min":1.332},"min":{"1min":0.777,"5min":0.722,"15min":0.722},"max":{"1min":2.257,"5min":7.975,"15min":7.975},"last":1.212}]},{"osd":3,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.515,"5min":8.301,"15min":3.706},"min":{"1min":0.720,"5min":0.659,"15min":0.659},"max":{"1min":8.877,"5min":660.469,"15min":660.469},"last":1.402},{"interface":"front","average":{"1min":36.220,"5min":8.352,"15min":3.754},"min":{"1min":0.784,"5min":0.784,"15min":0.784},"max":{"1min":455.681,"5min":455.681,"15min":455.681},"last":912.098}]},{"osd":4,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":0.989,"5min":33.395,"15min":47.786},"min":{"1min":0.562,"5min":0.537,"15min":0.537},"max":{"1min":1.550,"5min":886.107,"15min":886.107},"last":0.713},{"interface":"front","average":{"1min":1.078,"5min":1.266,"15min":1.349},"min":{"1min":0.667,"5min":0.526,"15min":0.526},"max":{"1min":1.415,"5min":11.065,"15min":11.065},"last":0.978}]},{"osd":6,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":14.251,"5min":3.795,"15min":2.053},"min":{"1min":0.933,"5min":0.583,"15min":0.583},"max":{"1min":260.820,"5min":260.820,"15min":260.820},"last":1.288},{"interface":"front","average":{"1min":1.293,"5min":1.245,"15min":1.237},"min":{"1min":0.679,"5min":0.612,"15min":0.612},"max":{"1min":2.292,"5min":2.292,"15min":2.292},"last":1.317}]},{"osd":7,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.257,"5min":1.237,"15min":1.233},"min":{"1min":0.891,"5min":0.841,"15min":0.841},"max":{"1min":1.672,"5min":1.672,"15min":1.672},"last":1.251},{"interface":"front","average":{"1min":1.465,"5min":1.333,"15min":1.311},"min":{"1min":1.148,"5min":0.802,"15min":0.802},"max":{"1min":2.166,"5min":2.175,"15min":2.175},"last":1.350}]},{"osd":8,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.484,"15min":1.476},"min":{"1min":0.982,"5min":0.982,"15min":0.982},"max":{"1min":2.315,"5min":2.315,"15min":2.315},"last":1.621},{"interface":"front","average":{"1min":1.618,"5min":5.511,"15min":6.160},"min":{"1min":1.196,"5min":1.196,"15min":1.196},"max":{"1min":2.334,"5min":91.314,"15min":91.314},"last":1.548}]},{"osd":9,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":1.204,"5min":1.204,"15min":1.204},"max":{"1min":1.880,"5min":1.880,"15min":1.880},"last":1.511},{"interface":"front","average":{"1min":1.543,"5min":1.543,"15min":1.543},"min":{"1min":1.177,"5min":1.177,"15min":1.177},"max":{"1min":2.430,"5min":2.430,"15min":2.430},"last":184.655}]},{"osd":10,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":1.135,"5min":1.135,"15min":1.135},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":1.475},{"interface":"front","average":{"1min":1.538,"5min":1.538,"15min":1.538},"min":{"1min":1.065,"5min":1.065,"15min":1.065},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.573}]}]}]}} 2019-12-22T22:28:10.019 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph pg dump --format=json 2019-12-22T22:28:12.337 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:28:12.337 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2019-12-22T22:28:12.873 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2019-12-22T22:28:10.916032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":12,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1124941824,"kb_used":12590400,"kb_used_data":7392,"kb_used_omap":0,"kb_used_meta":12582912,"kb_avail":1112351424,"statfs":{"total":1151940427776,"available":1139047858176,"internally_reserved":12884901888,"allocated":7569408,"data_stored":3466248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12884901888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[],"pool_stats":[],"osd_stats":[{"osd":11,"up_from":0,"seq":17,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.618,"5min":0.618,"15min":0.618},"max":{"1min":2.561,"5min":2.561,"15min":2.561},"last":1.189},{"interface":"front","average":{"1min":0.962,"5min":0.962,"15min":0.962},"min":{"1min":0.641,"5min":0.641,"15min":0.641},"max":{"1min":1.942,"5min":1.942,"15min":1.942},"last":0.847}]},{"osd":1,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":0.856,"5min":0.856,"15min":0.856},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.791},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.707,"5min":0.707,"15min":0.707},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":0.997}]},{"osd":2,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.037,"5min":1.037,"15min":1.037},"min":{"1min":0.812,"5min":0.812,"15min":0.812},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.037},{"interface":"front","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.656,"5min":0.656,"15min":0.656},"max":{"1min":1.312,"5min":1.312,"15min":1.312},"last":0.762}]},{"osd":3,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.687,"5min":0.687,"15min":0.687},"max":{"1min":1.666,"5min":1.666,"15min":1.666},"last":0.728},{"interface":"front","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.692,"5min":0.692,"15min":0.692},"max":{"1min":2.688,"5min":2.688,"15min":2.688},"last":0.907}]},{"osd":4,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.533,"5min":1.533,"15min":1.533},"min":{"1min":0.955,"5min":0.955,"15min":0.955},"max":{"1min":3.629,"5min":3.629,"15min":3.629},"last":1.397},{"interface":"front","average":{"1min":1.270,"5min":1.270,"15min":1.270},"min":{"1min":0.761,"5min":0.761,"15min":0.761},"max":{"1min":1.894,"5min":1.894,"15min":1.894},"last":1.133}]},{"osd":5,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.820,"5min":0.820,"15min":0.820},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.108},{"interface":"front","average":{"1min":1.686,"5min":1.686,"15min":1.686},"min":{"1min":1.128,"5min":1.128,"15min":1.128},"max":{"1min":3.739,"5min":3.739,"15min":3.739},"last":1.300}]},{"osd":6,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":35.379,"5min":35.379,"15min":35.379},"min":{"1min":0.984,"5min":0.984,"15min":0.984},"max":{"1min":646.716,"5min":646.716,"15min":646.716},"last":1.340},{"interface":"front","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.905,"5min":0.905,"15min":0.905},"max":{"1min":2.199,"5min":2.199,"15min":2.199},"last":0.968}]},{"osd":7,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.558,"5min":1.558,"15min":1.558},"min":{"1min":1.116,"5min":1.116,"15min":1.116},"max":{"1min":2.283,"5min":2.283,"15min":2.283},"last":1.442},{"interface":"front","average":{"1min":1.566,"5min":1.566,"15min":1.566},"min":{"1min":1.204,"5min":1.204,"15min":1.204},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":1.257}]},{"osd":8,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.822,"5min":1.822,"15min":1.822},"min":{"1min":1.237,"5min":1.237,"15min":1.237},"max":{"1min":3.936,"5min":3.936,"15min":3.936},"last":1.600},{"interface":"front","average":{"1min":34.440,"5min":34.440,"15min":34.440},"min":{"1min":1.332,"5min":1.332,"15min":1.332},"max":{"1min":624.904,"5min":624.904,"15min":624.904},"last":1.633}]},{"osd":10,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.871,"5min":1.871,"15min":1.871},"min":{"1min":1.205,"5min":1.205,"15min":1.205},"max":{"1min":4.041,"5min":4.041,"15min":4.041},"last":1.494},{"interface":"front","average":{"1min":1.772,"5min":1.772,"15min":1.772},"min":{"1min":1.354,"5min":1.354,"15min":1.354},"max":{"1min":3.823,"5min":3.823,"15min":3.823},"last":1.546}]}]},{"osd":10,"up_from":0,"seq":20,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":1.404}]},{"osd":1,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.667,"5min":0.667,"15min":0.667},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.619},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.712,"5min":0.712,"15min":0.712},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":1.832}]},{"osd":2,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.549,"5min":0.549,"15min":0.549},"max":{"1min":1.685,"5min":1.685,"15min":1.685},"last":1.343},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.604,"5min":0.604,"15min":0.604},"max":{"1min":2.191,"5min":2.191,"15min":2.191},"last":1.792}]},{"osd":3,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.010,"5min":1.010,"15min":1.010},"min":{"1min":0.635,"5min":0.635,"15min":0.635},"max":{"1min":1.438,"5min":1.438,"15min":1.438},"last":1.657},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.883,"5min":0.883,"15min":0.883},"max":{"1min":1.646,"5min":1.646,"15min":1.646},"last":1.484}]},{"osd":4,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.588,"5min":1.588,"15min":1.588},"min":{"1min":1.003,"5min":1.003,"15min":1.003},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":1.423},{"interface":"front","average":{"1min":1.915,"5min":1.915,"15min":1.915},"min":{"1min":0.740,"5min":0.740,"15min":0.740},"max":{"1min":11.454,"5min":11.454,"15min":11.454},"last":1.013}]},{"osd":5,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.430,"5min":1.430,"15min":1.430},"min":{"1min":0.757,"5min":0.757,"15min":0.757},"max":{"1min":2.450,"5min":2.450,"15min":2.450},"last":1.573},{"interface":"front","average":{"1min":1.681,"5min":1.681,"15min":1.681},"min":{"1min":1.204,"5min":1.204,"15min":1.204},"max":{"1min":2.777,"5min":2.777,"15min":2.777},"last":1.050}]},{"osd":6,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":2.012,"5min":2.012,"15min":2.012},"min":{"1min":1.031,"5min":1.031,"15min":1.031},"max":{"1min":11.404,"5min":11.404,"15min":11.404},"last":0.857},{"interface":"front","average":{"1min":1.559,"5min":1.559,"15min":1.559},"min":{"1min":1.108,"5min":1.108,"15min":1.108},"max":{"1min":2.462,"5min":2.462,"15min":2.462},"last":0.824}]},{"osd":7,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":102.420,"5min":102.420,"15min":102.420},"min":{"1min":1.064,"5min":1.064,"15min":1.064},"max":{"1min":938.067,"5min":938.067,"15min":938.067},"last":1.242},{"interface":"front","average":{"1min":1.644,"5min":1.644,"15min":1.644},"min":{"1min":1.151,"5min":1.151,"15min":1.151},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.972}]},{"osd":8,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":2.393,"5min":2.393,"15min":2.393},"min":{"1min":1.068,"5min":1.068,"15min":1.068},"max":{"1min":11.238,"5min":11.238,"15min":11.238},"last":1.300},{"interface":"front","average":{"1min":2.286,"5min":2.286,"15min":2.286},"min":{"1min":0.928,"5min":0.928,"15min":0.928},"max":{"1min":11.482,"5min":11.482,"15min":11.482},"last":1.373}]},{"osd":9,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":2.448,"5min":2.448,"15min":2.448},"min":{"1min":1.242,"5min":1.242,"15min":1.242},"max":{"1min":11.328,"5min":11.328,"15min":11.328},"last":1.210},{"interface":"front","average":{"1min":2.308,"5min":2.308,"15min":2.308},"min":{"1min":1.182,"5min":1.182,"15min":1.182},"max":{"1min":11.191,"5min":11.191,"15min":11.191},"last":1.522}]},{"osd":11,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.723,"5min":1.723,"15min":1.723},"min":{"1min":1.306,"5min":1.306,"15min":1.306},"max":{"1min":2.336,"5min":2.336,"15min":2.336},"last":1.150},{"interface":"front","average":{"1min":1.915,"5min":1.915,"15min":1.915},"min":{"1min":1.594,"5min":1.594,"15min":1.594},"max":{"1min":2.620,"5min":2.620,"15min":2.620},"last":1.696}]}]},{"osd":9,"up_from":0,"seq":25,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.084,"5min":1.084,"15min":1.084},"min":{"1min":0.636,"5min":0.636,"15min":0.636},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.961},{"interface":"front","average":{"1min":0.973,"5min":0.973,"15min":0.973},"min":{"1min":0.604,"5min":0.604,"15min":0.604},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.840}]},{"osd":1,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.037,"5min":1.037,"15min":1.037},"min":{"1min":0.656,"5min":0.656,"15min":0.656},"max":{"1min":1.664,"5min":1.664,"15min":1.664},"last":0.659},{"interface":"front","average":{"1min":0.891,"5min":0.891,"15min":0.891},"min":{"1min":0.647,"5min":0.647,"15min":0.647},"max":{"1min":1.437,"5min":1.437,"15min":1.437},"last":0.749}]},{"osd":2,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.197,"5min":1.197,"15min":1.197},"min":{"1min":0.738,"5min":0.738,"15min":0.738},"max":{"1min":2.162,"5min":2.162,"15min":2.162},"last":1.144},{"interface":"front","average":{"1min":27.186,"5min":27.186,"15min":27.186},"min":{"1min":0.668,"5min":0.668,"15min":0.668},"max":{"1min":524.094,"5min":524.094,"15min":524.094},"last":1.050}]},{"osd":3,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":35.175,"5min":35.175,"15min":35.175},"min":{"1min":0.538,"5min":0.538,"15min":0.538},"max":{"1min":683.851,"5min":683.851,"15min":683.851},"last":1.101},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.599,"5min":0.599,"15min":0.599},"max":{"1min":2.067,"5min":2.067,"15min":2.067},"last":1.302}]},{"osd":4,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.219,"5min":1.219,"15min":1.219},"min":{"1min":0.840,"5min":0.840,"15min":0.840},"max":{"1min":1.574,"5min":1.574,"15min":1.574},"last":1.208},{"interface":"front","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.716,"5min":0.716,"15min":0.716},"max":{"1min":1.720,"5min":1.720,"15min":1.720},"last":1.250}]},{"osd":5,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.275,"5min":1.275,"15min":1.275},"min":{"1min":0.693,"5min":0.693,"15min":0.693},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.460},{"interface":"front","average":{"1min":1.533,"5min":1.533,"15min":1.533},"min":{"1min":1.070,"5min":1.070,"15min":1.070},"max":{"1min":2.252,"5min":2.252,"15min":2.252},"last":1.523}]},{"osd":6,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":14.726,"5min":14.726,"15min":14.726},"min":{"1min":1.131,"5min":1.131,"15min":1.131},"max":{"1min":264.274,"5min":264.274,"15min":264.274},"last":1.611},{"interface":"front","average":{"1min":1.342,"5min":1.342,"15min":1.342},"min":{"1min":1.016,"5min":1.016,"15min":1.016},"max":{"1min":1.660,"5min":1.660,"15min":1.660},"last":1.352}]},{"osd":7,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.453,"5min":1.453,"15min":1.453},"min":{"1min":0.757,"5min":0.757,"15min":0.757},"max":{"1min":1.965,"5min":1.965,"15min":1.965},"last":1.420},{"interface":"front","average":{"1min":1.697,"5min":1.697,"15min":1.697},"min":{"1min":1.248,"5min":1.248,"15min":1.248},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":1.573}]},{"osd":8,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.443,"5min":1.443,"15min":1.443},"min":{"1min":0.700,"5min":0.700,"15min":0.700},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.495},{"interface":"front","average":{"1min":1.567,"5min":1.567,"15min":1.567},"min":{"1min":0.919,"5min":0.919,"15min":0.919},"max":{"1min":2.023,"5min":2.023,"15min":2.023},"last":1.662}]},{"osd":10,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.718,"5min":1.718,"15min":1.718},"min":{"1min":1.381,"5min":1.381,"15min":1.381},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":1.392},{"interface":"front","average":{"1min":1.802,"5min":1.802,"15min":1.802},"min":{"1min":1.407,"5min":1.407,"15min":1.407},"max":{"1min":2.188,"5min":2.188,"15min":2.188},"last":1.724}]}]},{"osd":8,"up_from":0,"seq":29,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.122,"5min":1.090,"15min":1.084},"min":{"1min":0.631,"5min":0.551,"15min":0.551},"max":{"1min":4.025,"5min":4.025,"15min":4.025},"last":1.894},{"interface":"front","average":{"1min":1.276,"5min":25.673,"15min":29.739},"min":{"1min":0.624,"5min":0.507,"15min":0.507},"max":{"1min":3.963,"5min":586.314,"15min":586.314},"last":1.834}]},{"osd":1,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":16.711,"5min":4.286,"15min":2.215},"min":{"1min":0.688,"5min":0.593,"15min":0.593},"max":{"1min":312.656,"5min":312.656,"15min":312.656},"last":1.717},{"interface":"front","average":{"1min":1.201,"5min":1.077,"15min":1.057},"min":{"1min":0.674,"5min":0.589,"15min":0.589},"max":{"1min":4.160,"5min":4.160,"15min":4.160},"last":1.649}]},{"osd":2,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.289,"5min":1.222,"15min":1.211},"min":{"1min":0.513,"5min":0.513,"15min":0.513},"max":{"1min":3.922,"5min":3.922,"15min":3.922},"last":2.009},{"interface":"front","average":{"1min":1.393,"5min":1.299,"15min":1.283},"min":{"1min":0.873,"5min":0.653,"15min":0.653},"max":{"1min":4.084,"5min":4.084,"15min":4.084},"last":1.948}]},{"osd":3,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.304,"15min":1.310},"min":{"1min":0.582,"5min":0.582,"15min":0.582},"max":{"1min":4.349,"5min":4.349,"15min":4.349},"last":1.488},{"interface":"front","average":{"1min":1.022,"5min":29.273,"15min":33.981},"min":{"1min":0.579,"5min":0.579,"15min":0.579},"max":{"1min":4.267,"5min":667.228,"15min":667.228},"last":1.778}]},{"osd":4,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.132,"5min":1.352,"15min":1.388},"min":{"1min":0.790,"5min":0.717,"15min":0.717},"max":{"1min":1.520,"5min":2.569,"15min":2.569},"last":1.372},{"interface":"front","average":{"1min":1.449,"5min":16.140,"15min":18.588},"min":{"1min":1.056,"5min":0.805,"15min":0.805},"max":{"1min":1.838,"5min":365.920,"15min":365.920},"last":1.320}]},{"osd":5,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.480,"15min":1.524},"min":{"1min":0.883,"5min":0.883,"15min":0.883},"max":{"1min":1.784,"5min":2.683,"15min":2.683},"last":1.010},{"interface":"front","average":{"1min":19.072,"5min":5.025,"15min":2.684},"min":{"1min":0.683,"5min":0.683,"15min":0.683},"max":{"1min":356.189,"5min":356.189,"15min":356.189},"last":0.954}]},{"osd":6,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.630,"15min":1.646},"min":{"1min":1.289,"5min":0.945,"15min":0.945},"max":{"1min":1.926,"5min":2.406,"15min":2.406},"last":1.147},{"interface":"front","average":{"1min":1.608,"5min":1.629,"15min":1.633},"min":{"1min":0.830,"5min":0.830,"15min":0.830},"max":{"1min":1.960,"5min":2.715,"15min":2.715},"last":1.279}]},{"osd":7,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":12.654,"15min":14.559},"min":{"1min":0.842,"5min":0.570,"15min":0.570},"max":{"1min":1.623,"5min":282.817,"15min":282.817},"last":1.211},{"interface":"front","average":{"1min":1.645,"5min":1.688,"15min":1.695},"min":{"1min":1.338,"5min":0.806,"15min":0.806},"max":{"1min":1.982,"5min":2.525,"15min":2.525},"last":1.419}]},{"osd":9,"last update":"Sun Dec 22 22:27:16 2019","interfaces":[{"interface":"back","average":{"1min":52.511,"5min":52.511,"15min":52.511},"min":{"1min":0.525,"5min":0.525,"15min":0.525},"max":{"1min":686.274,"5min":686.274,"15min":686.274},"last":0.701},{"interface":"front","average":{"1min":1.675,"5min":1.675,"15min":1.675},"min":{"1min":0.464,"5min":0.464,"15min":0.464},"max":{"1min":2.587,"5min":2.587,"15min":2.587},"last":1.089}]},{"osd":10,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":19.663,"5min":19.663,"15min":19.663},"min":{"1min":0.732,"5min":0.732,"15min":0.732},"max":{"1min":358.731,"5min":358.731,"15min":358.731},"last":1.734},{"interface":"front","average":{"1min":1.829,"5min":1.829,"15min":1.829},"min":{"1min":0.843,"5min":0.843,"15min":0.843},"max":{"1min":2.791,"5min":2.791,"15min":2.791},"last":1.547}]}]},{"osd":7,"up_from":0,"seq":32,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":0.973,"5min":0.854,"15min":0.834},"min":{"1min":0.558,"5min":0.558,"15min":0.558},"max":{"1min":1.443,"5min":1.443,"15min":1.443},"last":1.584},{"interface":"front","average":{"1min":14.982,"5min":3.818,"15min":1.958},"min":{"1min":0.731,"5min":0.684,"15min":0.684},"max":{"1min":291.908,"5min":291.908,"15min":291.908},"last":0.915}]},{"osd":1,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.197,"15min":1.201},"min":{"1min":0.819,"5min":0.758,"15min":0.758},"max":{"1min":1.616,"5min":2.308,"15min":2.308},"last":0.804},{"interface":"front","average":{"1min":1.079,"5min":1.059,"15min":1.056},"min":{"1min":0.680,"5min":0.680,"15min":0.680},"max":{"1min":1.396,"5min":1.937,"15min":1.937},"last":1.017}]},{"osd":2,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":0.991,"5min":1.018,"15min":1.022},"min":{"1min":0.685,"5min":0.685,"15min":0.685},"max":{"1min":1.529,"5min":1.886,"15min":1.886},"last":1.118},{"interface":"front","average":{"1min":1.121,"5min":1.145,"15min":1.149},"min":{"1min":0.817,"5min":0.625,"15min":0.625},"max":{"1min":1.524,"5min":2.376,"15min":2.376},"last":0.864}]},{"osd":3,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":42.500,"15min":49.405},"min":{"1min":0.648,"5min":0.648,"15min":0.648},"max":{"1min":1.494,"5min":985.876,"15min":985.876},"last":1.664},{"interface":"front","average":{"1min":1.217,"5min":1.272,"15min":1.281},"min":{"1min":0.784,"5min":0.694,"15min":0.694},"max":{"1min":1.559,"5min":2.153,"15min":2.153},"last":1.074}]},{"osd":4,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.143,"15min":1.151},"min":{"1min":0.699,"5min":0.600,"15min":0.600},"max":{"1min":1.654,"5min":1.800,"15min":1.800},"last":0.968},{"interface":"front","average":{"1min":1.159,"5min":1.191,"15min":1.196},"min":{"1min":0.704,"5min":0.704,"15min":0.704},"max":{"1min":1.643,"5min":1.833,"15min":1.833},"last":1.446}]},{"osd":5,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.154,"5min":1.273,"15min":1.293},"min":{"1min":0.633,"5min":0.633,"15min":0.633},"max":{"1min":1.629,"5min":1.663,"15min":1.663},"last":1.321},{"interface":"front","average":{"1min":1.398,"5min":1.387,"15min":1.385},"min":{"1min":0.855,"5min":0.728,"15min":0.728},"max":{"1min":1.747,"5min":2.435,"15min":2.435},"last":1.260}]},{"osd":6,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.322,"5min":1.273,"15min":1.265},"min":{"1min":0.611,"5min":0.611,"15min":0.611},"max":{"1min":1.710,"5min":2.106,"15min":2.106},"last":1.364},{"interface":"front","average":{"1min":1.369,"5min":31.793,"15min":36.864},"min":{"1min":0.976,"5min":0.976,"15min":0.976},"max":{"1min":1.755,"5min":684.801,"15min":684.801},"last":1.540}]},{"osd":8,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":29.540,"5min":7.085,"15min":3.343},"min":{"1min":0.901,"5min":0.807,"15min":0.807},"max":{"1min":502.217,"5min":502.217,"15min":502.217},"last":1.745},{"interface":"front","average":{"1min":1.560,"5min":1.628,"15min":1.639},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":1.788,"5min":2.619,"15min":2.619},"last":1.501}]},{"osd":9,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.606,"5min":1.606,"15min":1.606},"min":{"1min":1.228,"5min":1.228,"15min":1.228},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.707},{"interface":"front","average":{"1min":46.020,"5min":46.020,"15min":46.020},"min":{"1min":1.173,"5min":1.173,"15min":1.173},"max":{"1min":891.130,"5min":891.130,"15min":891.130},"last":1.816}]},{"osd":10,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":1.671,"15min":1.671},"min":{"1min":1.193,"5min":1.193,"15min":1.193},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.395},{"interface":"front","average":{"1min":53.065,"5min":53.065,"15min":53.065},"min":{"1min":1.327,"5min":1.327,"15min":1.327},"max":{"1min":748.773,"5min":748.773,"15min":748.773},"last":1.628}]}]},{"osd":6,"up_from":0,"seq":38,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":1.053,"15min":1.083},"min":{"1min":0.601,"5min":0.601,"15min":0.601},"max":{"1min":1.682,"5min":1.682,"15min":1.682},"last":1.172},{"interface":"front","average":{"1min":0.974,"5min":1.074,"15min":1.090},"min":{"1min":0.689,"5min":0.689,"15min":0.689},"max":{"1min":1.528,"5min":1.677,"15min":1.677},"last":0.900}]},{"osd":1,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.893,"15min":0.895},"min":{"1min":0.626,"5min":0.591,"15min":0.591},"max":{"1min":1.364,"5min":1.793,"15min":1.793},"last":1.214},{"interface":"front","average":{"1min":0.920,"5min":19.041,"15min":22.061},"min":{"1min":0.611,"5min":0.588,"15min":0.588},"max":{"1min":1.628,"5min":363.766,"15min":363.766},"last":1.048}]},{"osd":2,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":5.483,"5min":1.961,"15min":1.374},"min":{"1min":0.674,"5min":0.674,"15min":0.674},"max":{"1min":89.000,"5min":89.000,"15min":89.000},"last":0.839},{"interface":"front","average":{"1min":1.033,"5min":46.178,"15min":53.702},"min":{"1min":0.721,"5min":0.721,"15min":0.721},"max":{"1min":1.550,"5min":904.141,"15min":904.141},"last":0.734}]},{"osd":3,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":16.675,"15min":19.261},"min":{"1min":0.902,"5min":0.834,"15min":0.834},"max":{"1min":1.511,"5min":312.765,"15min":312.765},"last":1.005},{"interface":"front","average":{"1min":1.160,"5min":1.184,"15min":1.188},"min":{"1min":0.719,"5min":0.719,"15min":0.719},"max":{"1min":1.468,"5min":1.771,"15min":1.771},"last":1.287}]},{"osd":4,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":47.470,"5min":10.303,"15min":4.109},"min":{"1min":0.727,"5min":0.535,"15min":0.535},"max":{"1min":882.404,"5min":882.404,"15min":882.404},"last":0.797},{"interface":"front","average":{"1min":1.110,"5min":1.036,"15min":1.024},"min":{"1min":0.653,"5min":0.595,"15min":0.595},"max":{"1min":1.572,"5min":1.572,"15min":1.572},"last":1.093}]},{"osd":5,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.197,"5min":1.164,"15min":1.158},"min":{"1min":0.843,"5min":0.680,"15min":0.680},"max":{"1min":1.453,"5min":1.839,"15min":1.839},"last":1.260},{"interface":"front","average":{"1min":1.296,"5min":1.257,"15min":1.251},"min":{"1min":0.960,"5min":0.896,"15min":0.896},"max":{"1min":1.706,"5min":1.804,"15min":1.804},"last":1.326}]},{"osd":7,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.311,"15min":1.307},"min":{"1min":1.073,"5min":1.030,"15min":1.030},"max":{"1min":1.590,"5min":1.675,"15min":1.675},"last":1.423},{"interface":"front","average":{"1min":1.392,"5min":1.350,"15min":1.343},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.656,"5min":1.721,"15min":1.721},"last":1.450}]},{"osd":8,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":49.165,"15min":57.102},"min":{"1min":1.168,"5min":1.168,"15min":1.168},"max":{"1min":1.972,"5min":844.789,"15min":844.789},"last":1.554},{"interface":"front","average":{"1min":1.408,"5min":1.431,"15min":1.435},"min":{"1min":1.200,"5min":1.183,"15min":1.183},"max":{"1min":1.840,"5min":1.840,"15min":1.840},"last":1.373}]},{"osd":9,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.531,"5min":1.531,"15min":1.531},"min":{"1min":1.113,"5min":1.113,"15min":1.113},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":1.596},{"interface":"front","average":{"1min":1.485,"5min":1.485,"15min":1.485},"min":{"1min":1.043,"5min":1.043,"15min":1.043},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.502}]},{"osd":10,"last update":"Sun Dec 22 22:27:37 2019","interfaces":[{"interface":"back","average":{"1min":1.497,"5min":1.497,"15min":1.497},"min":{"1min":1.156,"5min":1.156,"15min":1.156},"max":{"1min":1.866,"5min":1.866,"15min":1.866},"last":1.530},{"interface":"front","average":{"1min":1.596,"5min":1.596,"15min":1.596},"min":{"1min":1.000,"5min":1.000,"15min":1.000},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.625}]}]},{"osd":1,"up_from":0,"seq":58,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":0.805,"5min":0.645,"15min":0.595},"min":{"1min":0.525,"5min":0.338,"15min":0.338},"max":{"1min":1.279,"5min":1.279,"15min":1.279},"last":0.879},{"interface":"front","average":{"1min":0.901,"5min":0.829,"15min":0.735},"min":{"1min":0.507,"5min":0.381,"15min":0.381},"max":{"1min":1.370,"5min":1.511,"15min":1.511},"last":1.488}]},{"osd":2,"last update":"Sun Dec 22 22:28:05 2019","interfaces":[{"interface":"back","average":{"1min":1.011,"5min":0.885,"15min":0.797},"min":{"1min":0.500,"5min":0.391,"15min":0.391},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.161},{"interface":"front","average":{"1min":0.869,"5min":3.282,"15min":1.544},"min":{"1min":0.475,"5min":0.436,"15min":0.436},"max":{"1min":1.471,"5min":254.847,"15min":254.847},"last":0.817}]},{"osd":3,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":46.363,"5min":18.287,"15min":6.627},"min":{"1min":0.651,"5min":0.551,"15min":0.551},"max":{"1min":952.051,"5min":952.051,"15min":952.051},"last":1.299},{"interface":"front","average":{"1min":1.186,"5min":17.623,"15min":14.386},"min":{"1min":0.620,"5min":0.620,"15min":0.620},"max":{"1min":1.684,"5min":903.487,"15min":903.487},"last":1.549}]},{"osd":4,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":1.010,"5min":0.878,"15min":0.847},"min":{"1min":0.707,"5min":0.601,"15min":0.601},"max":{"1min":1.424,"5min":1.424,"15min":1.424},"last":0.965},{"interface":"front","average":{"1min":1.338,"5min":1.193,"15min":1.117},"min":{"1min":0.946,"5min":0.732,"15min":0.732},"max":{"1min":1.685,"5min":1.941,"15min":1.941},"last":1.592}]},{"osd":5,"last update":"Sun Dec 22 22:28:02 2019","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":0.977,"15min":0.957},"min":{"1min":0.851,"5min":0.676,"15min":0.676},"max":{"1min":1.521,"5min":1.521,"15min":1.521},"last":1.091},{"interface":"front","average":{"1min":12.769,"5min":3.568,"15min":2.000},"min":{"1min":0.809,"5min":0.809,"15min":0.809},"max":{"1min":263.068,"5min":263.068,"15min":263.068},"last":1.669}]},{"osd":6,"last update":"Sun Dec 22 22:27:23 2019","interfaces":[{"interface":"back","average":{"1min":1.410,"5min":1.411,"15min":1.411},"min":{"1min":0.972,"5min":0.972,"15min":0.972},"max":{"1min":2.000,"5min":2.315,"15min":2.315},"last":1.636},{"interface":"front","average":{"1min":1.019,"5min":1.013,"15min":1.012},"min":{"1min":0.736,"5min":0.736,"15min":0.736},"max":{"1min":1.527,"5min":1.673,"15min":1.673},"last":1.207}]},{"osd":7,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":1.400,"5min":1.473,"15min":1.485},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.861,"5min":2.238,"15min":2.238},"last":1.877},{"interface":"front","average":{"1min":6.449,"5min":2.532,"15min":1.879},"min":{"1min":0.963,"5min":0.963,"15min":0.963},"max":{"1min":101.788,"5min":101.788,"15min":101.788},"last":1.852}]},{"osd":8,"last update":"Sun Dec 22 22:28:02 2019","interfaces":[{"interface":"back","average":{"1min":1.189,"5min":1.189,"15min":1.189},"min":{"1min":0.826,"5min":0.826,"15min":0.826},"max":{"1min":1.594,"5min":1.594,"15min":1.594},"last":1.255},{"interface":"front","average":{"1min":1.196,"5min":1.228,"15min":1.233},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":1.372}]},{"osd":9,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.589,"5min":1.589,"15min":1.589},"min":{"1min":1.121,"5min":1.121,"15min":1.121},"max":{"1min":2.231,"5min":2.231,"15min":2.231},"last":1.815},{"interface":"front","average":{"1min":15.510,"5min":15.510,"15min":15.510},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":300.557,"5min":300.557,"15min":300.557},"last":1.524}]},{"osd":10,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":46.030,"5min":46.030,"15min":46.030},"min":{"1min":1.021,"5min":1.021,"15min":1.021},"max":{"1min":896.161,"5min":896.161,"15min":896.161},"last":1.451},{"interface":"front","average":{"1min":1.528,"5min":1.528,"15min":1.528},"min":{"1min":1.144,"5min":1.144,"15min":1.144},"max":{"1min":1.902,"5min":1.902,"15min":1.902},"last":1.774}]}]},{"osd":0,"up_from":0,"seq":63,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":0.912,"5min":0.792,"15min":0.752},"min":{"1min":0.582,"5min":0.291,"15min":0.291},"max":{"1min":1.381,"5min":1.720,"15min":1.720},"last":1.089},{"interface":"front","average":{"1min":1.068,"5min":0.864,"15min":0.760},"min":{"1min":0.659,"5min":0.349,"15min":0.349},"max":{"1min":1.570,"5min":1.597,"15min":1.597},"last":1.321}]},{"osd":2,"last update":"Sun Dec 22 22:26:03 2019","interfaces":[{"interface":"back","average":{"1min":0.938,"5min":0.922,"15min":0.919},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.347,"5min":1.798,"15min":1.798},"last":1.188},{"interface":"front","average":{"1min":38.337,"5min":38.242,"15min":38.226},"min":{"1min":0.580,"5min":0.552,"15min":0.552},"max":{"1min":601.565,"5min":601.565,"15min":601.565},"last":0.645}]},{"osd":3,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.047,"5min":0.844,"15min":0.776},"min":{"1min":0.616,"5min":0.540,"15min":0.540},"max":{"1min":1.722,"5min":1.832,"15min":1.832},"last":1.031},{"interface":"front","average":{"1min":0.724,"5min":12.273,"15min":4.552},"min":{"1min":0.525,"5min":0.525,"15min":0.525},"max":{"1min":1.155,"5min":867.546,"15min":867.546},"last":0.936}]},{"osd":4,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":0.927,"5min":0.888,"15min":0.848},"min":{"1min":0.673,"5min":0.664,"15min":0.664},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":1.555},{"interface":"front","average":{"1min":1.313,"5min":1.149,"15min":1.077},"min":{"1min":0.945,"5min":0.850,"15min":0.850},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":1.442}]},{"osd":5,"last update":"Sun Dec 22 22:28:04 2019","interfaces":[{"interface":"back","average":{"1min":1.273,"5min":1.207,"15min":1.156},"min":{"1min":0.875,"5min":0.802,"15min":0.802},"max":{"1min":1.922,"5min":1.922,"15min":1.922},"last":1.758},{"interface":"front","average":{"1min":0.907,"5min":0.982,"15min":0.970},"min":{"1min":0.654,"5min":0.654,"15min":0.654},"max":{"1min":1.647,"5min":1.887,"15min":1.887},"last":1.201}]},{"osd":6,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.073,"15min":1.070},"min":{"1min":0.709,"5min":0.709,"15min":0.709},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":1.413},{"interface":"front","average":{"1min":1.480,"5min":26.547,"15min":30.725},"min":{"1min":0.923,"5min":0.923,"15min":0.923},"max":{"1min":2.246,"5min":504.570,"15min":504.570},"last":1.509}]},{"osd":7,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.195,"15min":1.199},"min":{"1min":0.863,"5min":0.863,"15min":0.863},"max":{"1min":1.616,"5min":1.616,"15min":1.616},"last":1.478},{"interface":"front","average":{"1min":1.488,"5min":26.395,"15min":30.546},"min":{"1min":0.975,"5min":0.975,"15min":0.975},"max":{"1min":1.885,"5min":530.298,"15min":530.298},"last":1.670}]},{"osd":8,"last update":"Sun Dec 22 22:28:04 2019","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.303,"15min":1.319},"min":{"1min":0.815,"5min":0.805,"15min":0.805},"max":{"1min":1.771,"5min":1.771,"15min":1.771},"last":1.642},{"interface":"front","average":{"1min":46.831,"5min":10.652,"15min":4.622},"min":{"1min":0.892,"5min":0.892,"15min":0.892},"max":{"1min":817.964,"5min":817.964,"15min":817.964},"last":1.902}]},{"osd":9,"last update":"Sun Dec 22 22:27:17 2019","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.649,"15min":1.649},"min":{"1min":1.280,"5min":1.280,"15min":1.280},"max":{"1min":2.180,"5min":2.180,"15min":2.180},"last":1.865},{"interface":"front","average":{"1min":1.264,"5min":1.264,"15min":1.264},"min":{"1min":0.969,"5min":0.969,"15min":0.969},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.278}]},{"osd":10,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.639,"15min":1.639},"min":{"1min":1.130,"5min":1.130,"15min":1.130},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":1.798},{"interface":"front","average":{"1min":1.644,"5min":1.644,"15min":1.644},"min":{"1min":1.072,"5min":1.072,"15min":1.072},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":1.609}]},{"osd":11,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.160,"5min":1.160,"15min":1.160},"min":{"1min":0.772,"5min":0.772,"15min":0.772},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.364},{"interface":"front","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":0.853,"5min":0.853,"15min":0.853},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.714}]}]},{"osd":2,"up_from":0,"seq":55,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:28:07 2019","interfaces":[{"interface":"back","average":{"1min":0.733,"5min":0.657,"15min":0.567},"min":{"1min":0.418,"5min":0.398,"15min":0.398},"max":{"1min":1.422,"5min":1.461,"15min":1.461},"last":0.807},{"interface":"front","average":{"1min":0.832,"5min":0.699,"15min":0.623},"min":{"1min":0.423,"5min":0.423,"15min":0.423},"max":{"1min":1.684,"5min":1.684,"15min":1.684},"last":0.929}]},{"osd":1,"last update":"Sun Dec 22 22:28:07 2019","interfaces":[{"interface":"back","average":{"1min":0.909,"5min":0.819,"15min":0.745},"min":{"1min":0.586,"5min":0.501,"15min":0.501},"max":{"1min":1.324,"5min":1.555,"15min":1.555},"last":1.137},{"interface":"front","average":{"1min":1.038,"5min":0.810,"15min":0.744},"min":{"1min":0.656,"5min":0.478,"15min":0.478},"max":{"1min":1.406,"5min":1.406,"15min":1.406},"last":1.526}]},{"osd":3,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.029,"5min":0.868,"15min":0.818},"min":{"1min":0.711,"5min":0.563,"15min":0.563},"max":{"1min":1.261,"5min":1.261,"15min":1.261},"last":1.263},{"interface":"front","average":{"1min":0.952,"5min":0.840,"15min":0.818},"min":{"1min":0.611,"5min":0.525,"15min":0.525},"max":{"1min":1.930,"5min":1.930,"15min":1.930},"last":0.875}]},{"osd":4,"last update":"Sun Dec 22 22:27:42 2019","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.019,"15min":0.995},"min":{"1min":0.594,"5min":0.594,"15min":0.594},"max":{"1min":1.947,"5min":1.947,"15min":1.947},"last":1.043},{"interface":"front","average":{"1min":1.112,"5min":1.104,"15min":1.094},"min":{"1min":0.690,"5min":0.668,"15min":0.668},"max":{"1min":1.588,"5min":1.588,"15min":1.588},"last":1.440}]},{"osd":5,"last update":"Sun Dec 22 22:28:07 2019","interfaces":[{"interface":"back","average":{"1min":1.142,"5min":1.049,"15min":1.016},"min":{"1min":0.791,"5min":0.689,"15min":0.689},"max":{"1min":1.552,"5min":1.585,"15min":1.585},"last":1.609},{"interface":"front","average":{"1min":1.199,"5min":1.159,"15min":1.135},"min":{"1min":0.744,"5min":0.744,"15min":0.744},"max":{"1min":2.460,"5min":2.460,"15min":2.460},"last":1.313}]},{"osd":6,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.116,"5min":1.071,"15min":1.063},"min":{"1min":0.707,"5min":0.707,"15min":0.707},"max":{"1min":1.998,"5min":2.034,"15min":2.034},"last":1.103},{"interface":"front","average":{"1min":1.232,"5min":23.528,"15min":27.244},"min":{"1min":0.793,"5min":0.793,"15min":0.793},"max":{"1min":2.354,"5min":587.877,"15min":587.877},"last":1.484}]},{"osd":7,"last update":"Sun Dec 22 22:27:42 2019","interfaces":[{"interface":"back","average":{"1min":1.180,"5min":1.195,"15min":1.197},"min":{"1min":0.730,"5min":0.730,"15min":0.730},"max":{"1min":2.066,"5min":2.066,"15min":2.066},"last":1.223},{"interface":"front","average":{"1min":1.314,"5min":1.283,"15min":1.278},"min":{"1min":0.780,"5min":0.780,"15min":0.780},"max":{"1min":2.096,"5min":2.096,"15min":2.096},"last":1.569}]},{"osd":8,"last update":"Sun Dec 22 22:27:57 2019","interfaces":[{"interface":"back","average":{"1min":37.270,"5min":8.562,"15min":3.777},"min":{"1min":0.910,"5min":0.910,"15min":0.910},"max":{"1min":648.474,"5min":648.474,"15min":648.474},"last":1.711},{"interface":"front","average":{"1min":40.243,"5min":9.160,"15min":3.980},"min":{"1min":0.690,"5min":0.690,"15min":0.690},"max":{"1min":705.146,"5min":705.146,"15min":705.146},"last":1.352}]},{"osd":9,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":47.812,"5min":47.812,"15min":47.812},"min":{"1min":1.126,"5min":1.126,"15min":1.126},"max":{"1min":834.497,"5min":834.497,"15min":834.497},"last":1.742},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":1.017,"5min":1.017,"15min":1.017},"max":{"1min":1.749,"5min":1.749,"15min":1.749},"last":1.828}]},{"osd":10,"last update":"Sun Dec 22 22:27:42 2019","interfaces":[{"interface":"back","average":{"1min":59.261,"5min":59.261,"15min":59.261},"min":{"1min":1.117,"5min":1.117,"15min":1.117},"max":{"1min":984.644,"5min":984.644,"15min":984.644},"last":1.675},{"interface":"front","average":{"1min":1.477,"5min":1.477,"15min":1.477},"min":{"1min":0.897,"5min":0.897,"15min":0.897},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.775}]}]},{"osd":3,"up_from":0,"seq":52,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:26:18 2019","interfaces":[{"interface":"back","average":{"1min":0.828,"5min":35.683,"15min":41.492},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.248,"5min":961.866,"15min":961.866},"last":1.531},{"interface":"front","average":{"1min":0.810,"5min":0.701,"15min":0.683},"min":{"1min":0.510,"5min":0.510,"15min":0.510},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.753}]},{"osd":1,"last update":"Sun Dec 22 22:27:21 2019","interfaces":[{"interface":"back","average":{"1min":0.865,"5min":0.740,"15min":0.674},"min":{"1min":0.512,"5min":0.498,"15min":0.498},"max":{"1min":1.178,"5min":2.091,"15min":2.091},"last":0.496},{"interface":"front","average":{"1min":28.452,"5min":6.305,"15min":2.590},"min":{"1min":0.407,"5min":0.407,"15min":0.407},"max":{"1min":495.556,"5min":495.556,"15min":495.556},"last":0.465}]},{"osd":2,"last update":"Sun Dec 22 22:27:21 2019","interfaces":[{"interface":"back","average":{"1min":5.722,"5min":23.239,"15min":32.091},"min":{"1min":0.473,"5min":0.473,"15min":0.473},"max":{"1min":87.773,"5min":750.955,"15min":750.955},"last":0.554},{"interface":"front","average":{"1min":49.085,"5min":10.532,"15min":4.098},"min":{"1min":0.541,"5min":0.541,"15min":0.541},"max":{"1min":866.045,"5min":866.045,"15min":866.045},"last":0.618}]},{"osd":4,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.186,"15min":1.186},"min":{"1min":0.730,"5min":0.730,"15min":0.730},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.064},{"interface":"front","average":{"1min":0.900,"5min":1.007,"15min":1.023},"min":{"1min":0.663,"5min":0.663,"15min":0.663},"max":{"1min":1.231,"5min":1.657,"15min":1.657},"last":0.717}]},{"osd":5,"last update":"Sun Dec 22 22:28:00 2019","interfaces":[{"interface":"back","average":{"1min":39.628,"5min":8.791,"15min":3.613},"min":{"1min":0.574,"5min":0.574,"15min":0.574},"max":{"1min":501.574,"5min":501.574,"15min":501.574},"last":0.678},{"interface":"front","average":{"1min":1.165,"5min":1.205,"15min":1.201},"min":{"1min":0.678,"5min":0.678,"15min":0.678},"max":{"1min":1.649,"5min":1.845,"15min":1.845},"last":1.164}]},{"osd":6,"last update":"Sun Dec 22 22:27:21 2019","interfaces":[{"interface":"back","average":{"1min":28.463,"5min":6.643,"15min":3.007},"min":{"1min":0.650,"5min":0.650,"15min":0.650},"max":{"1min":246.895,"5min":246.895,"15min":246.895},"last":0.806},{"interface":"front","average":{"1min":11.539,"5min":3.329,"15min":1.961},"min":{"1min":0.597,"5min":0.597,"15min":0.597},"max":{"1min":184.604,"5min":184.604,"15min":184.604},"last":0.767}]},{"osd":7,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":14.350,"15min":16.570},"min":{"1min":0.802,"5min":0.768,"15min":0.768},"max":{"1min":1.493,"5min":347.131,"15min":347.131},"last":1.035},{"interface":"front","average":{"1min":1.230,"5min":1.334,"15min":1.352},"min":{"1min":0.775,"5min":0.775,"15min":0.775},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.744}]},{"osd":8,"last update":"Sun Dec 22 22:28:00 2019","interfaces":[{"interface":"back","average":{"1min":1.372,"5min":1.411,"15min":1.417},"min":{"1min":0.885,"5min":0.885,"15min":0.885},"max":{"1min":2.279,"5min":2.279,"15min":2.279},"last":1.143},{"interface":"front","average":{"1min":1.366,"5min":1.420,"15min":1.429},"min":{"1min":0.865,"5min":0.865,"15min":0.865},"max":{"1min":1.838,"5min":2.137,"15min":2.137},"last":1.232}]},{"osd":9,"last update":"Sun Dec 22 22:27:21 2019","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":1.001,"5min":1.001,"15min":1.001},"max":{"1min":1.658,"5min":1.658,"15min":1.658},"last":1.105},{"interface":"front","average":{"1min":1.489,"5min":1.489,"15min":1.489},"min":{"1min":1.019,"5min":1.019,"15min":1.019},"max":{"1min":2.025,"5min":2.025,"15min":2.025},"last":0.995}]},{"osd":10,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":6.989,"5min":6.989,"15min":6.989},"min":{"1min":0.978,"5min":0.978,"15min":0.978},"max":{"1min":110.434,"5min":110.434,"15min":110.434},"last":0.931},{"interface":"front","average":{"1min":1.545,"5min":1.545,"15min":1.545},"min":{"1min":0.928,"5min":0.928,"15min":0.928},"max":{"1min":7.829,"5min":7.829,"15min":7.829},"last":0.859}]},{"osd":11,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.441,"5min":1.441,"15min":1.441},"min":{"1min":0.952,"5min":0.952,"15min":0.952},"max":{"1min":2.313,"5min":2.313,"15min":2.313},"last":1.189},{"interface":"front","average":{"1min":1.445,"5min":1.445,"15min":1.445},"min":{"1min":1.052,"5min":1.052,"15min":1.052},"max":{"1min":1.781,"5min":1.781,"15min":1.781},"last":1.266}]}]},{"osd":4,"up_from":0,"seq":47,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":43.486,"5min":9.677,"15min":3.841},"min":{"1min":0.734,"5min":0.627,"15min":0.627},"max":{"1min":764.951,"5min":764.951,"15min":764.951},"last":0.943},{"interface":"front","average":{"1min":1.035,"5min":26.738,"15min":31.536},"min":{"1min":0.685,"5min":0.614,"15min":0.614},"max":{"1min":1.430,"5min":660.413,"15min":660.413},"last":0.748}]},{"osd":1,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":1.108,"5min":0.943,"15min":0.901},"min":{"1min":0.884,"5min":0.585,"15min":0.585},"max":{"1min":1.451,"5min":1.451,"15min":1.451},"last":1.343},{"interface":"front","average":{"1min":0.897,"5min":0.912,"15min":0.916},"min":{"1min":0.645,"5min":0.626,"15min":0.626},"max":{"1min":1.192,"5min":1.335,"15min":1.335},"last":1.006}]},{"osd":2,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":0.913,"5min":0.923,"15min":0.911},"min":{"1min":0.698,"5min":0.677,"15min":0.677},"max":{"1min":1.124,"5min":1.223,"15min":1.223},"last":0.795},{"interface":"front","average":{"1min":1.141,"5min":1.052,"15min":1.010},"min":{"1min":0.613,"5min":0.613,"15min":0.613},"max":{"1min":1.660,"5min":1.993,"15min":1.993},"last":0.582}]},{"osd":3,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":21.111,"5min":20.868,"15min":25.206},"min":{"1min":0.684,"5min":0.577,"15min":0.577},"max":{"1min":361.413,"5min":503.582,"15min":503.582},"last":0.687},{"interface":"front","average":{"1min":1.010,"5min":0.978,"15min":0.947},"min":{"1min":0.738,"5min":0.585,"15min":0.585},"max":{"1min":1.347,"5min":1.496,"15min":1.496},"last":0.896}]},{"osd":5,"last update":"Sun Dec 22 22:27:57 2019","interfaces":[{"interface":"back","average":{"1min":10.611,"5min":7.111,"15min":3.013},"min":{"1min":0.672,"5min":0.477,"15min":0.477},"max":{"1min":200.826,"5min":357.229,"15min":357.229},"last":1.235},{"interface":"front","average":{"1min":0.957,"5min":0.918,"15min":0.910},"min":{"1min":0.632,"5min":0.456,"15min":0.456},"max":{"1min":1.204,"5min":1.359,"15min":1.359},"last":1.135}]},{"osd":6,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":3.086,"15min":3.410},"min":{"1min":0.585,"5min":0.495,"15min":0.495},"max":{"1min":1.708,"5min":49.709,"15min":49.709},"last":1.095},{"interface":"front","average":{"1min":1.371,"5min":31.997,"15min":37.101},"min":{"1min":0.889,"5min":0.734,"15min":0.734},"max":{"1min":1.732,"5min":768.708,"15min":768.708},"last":1.179}]},{"osd":7,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.431,"15min":1.431},"min":{"1min":1.133,"5min":1.004,"15min":1.004},"max":{"1min":1.766,"5min":2.058,"15min":2.058},"last":1.287},{"interface":"front","average":{"1min":1.409,"5min":1.289,"15min":1.269},"min":{"1min":1.193,"5min":0.814,"15min":0.814},"max":{"1min":1.638,"5min":1.695,"15min":1.695},"last":1.050}]},{"osd":8,"last update":"Sun Dec 22 22:27:57 2019","interfaces":[{"interface":"back","average":{"1min":35.376,"5min":8.328,"15min":3.820},"min":{"1min":1.133,"5min":1.133,"15min":1.133},"max":{"1min":678.500,"5min":678.500,"15min":678.500},"last":1.555},{"interface":"front","average":{"1min":42.957,"5min":9.778,"15min":4.248},"min":{"1min":1.182,"5min":1.055,"15min":1.055},"max":{"1min":831.286,"5min":831.286,"15min":831.286},"last":1.638}]},{"osd":9,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.529,"5min":1.529,"15min":1.529},"min":{"1min":1.310,"5min":1.310,"15min":1.310},"max":{"1min":1.869,"5min":1.869,"15min":1.869},"last":1.404},{"interface":"front","average":{"1min":1.480,"5min":1.480,"15min":1.480},"min":{"1min":1.227,"5min":1.227,"15min":1.227},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":1.506}]},{"osd":10,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":18.270,"5min":18.270,"15min":18.270},"min":{"1min":1.313,"5min":1.313,"15min":1.313},"max":{"1min":302.146,"5min":302.146,"15min":302.146},"last":1.462},{"interface":"front","average":{"1min":1.541,"5min":1.541,"15min":1.541},"min":{"1min":1.340,"5min":1.340,"15min":1.340},"max":{"1min":1.767,"5min":1.767,"15min":1.767},"last":1.595}]}]},{"osd":5,"up_from":0,"seq":42,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.189,"15min":1.258},"min":{"1min":0.611,"5min":0.557,"15min":0.557},"max":{"1min":2.727,"5min":7.646,"15min":7.646},"last":1.431},{"interface":"front","average":{"1min":1.145,"5min":22.091,"15min":31.454},"min":{"1min":0.743,"5min":0.548,"15min":0.548},"max":{"1min":2.694,"5min":803.966,"15min":803.966},"last":1.125}]},{"osd":1,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":12.150,"15min":17.025},"min":{"1min":0.679,"5min":0.553,"15min":0.553},"max":{"1min":8.832,"5min":456.064,"15min":456.064},"last":1.046},{"interface":"front","average":{"1min":1.067,"5min":1.152,"15min":1.219},"min":{"1min":0.632,"5min":0.619,"15min":0.619},"max":{"1min":1.543,"5min":7.728,"15min":7.728},"last":0.916}]},{"osd":2,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.406,"5min":8.846,"15min":12.250},"min":{"1min":0.658,"5min":0.651,"15min":0.651},"max":{"1min":8.927,"5min":319.127,"15min":319.127},"last":1.177},{"interface":"front","average":{"1min":1.141,"5min":1.268,"15min":1.332},"min":{"1min":0.777,"5min":0.722,"15min":0.722},"max":{"1min":2.257,"5min":7.975,"15min":7.975},"last":1.212}]},{"osd":3,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.515,"5min":8.301,"15min":3.706},"min":{"1min":0.720,"5min":0.659,"15min":0.659},"max":{"1min":8.877,"5min":660.469,"15min":660.469},"last":1.402},{"interface":"front","average":{"1min":36.220,"5min":8.352,"15min":3.754},"min":{"1min":0.784,"5min":0.784,"15min":0.784},"max":{"1min":455.681,"5min":455.681,"15min":455.681},"last":912.098}]},{"osd":4,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":0.989,"5min":33.395,"15min":47.786},"min":{"1min":0.562,"5min":0.537,"15min":0.537},"max":{"1min":1.550,"5min":886.107,"15min":886.107},"last":0.713},{"interface":"front","average":{"1min":1.078,"5min":1.266,"15min":1.349},"min":{"1min":0.667,"5min":0.526,"15min":0.526},"max":{"1min":1.415,"5min":11.065,"15min":11.065},"last":0.978}]},{"osd":6,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":14.251,"5min":3.795,"15min":2.053},"min":{"1min":0.933,"5min":0.583,"15min":0.583},"max":{"1min":260.820,"5min":260.820,"15min":260.820},"last":1.288},{"interface":"front","average":{"1min":1.293,"5min":1.245,"15min":1.237},"min":{"1min":0.679,"5min":0.612,"15min":0.612},"max":{"1min":2.292,"5min":2.292,"15min":2.292},"last":1.317}]},{"osd":7,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.257,"5min":1.237,"15min":1.233},"min":{"1min":0.891,"5min":0.841,"15min":0.841},"max":{"1min":1.672,"5min":1.672,"15min":1.672},"last":1.251},{"interface":"front","average":{"1min":1.465,"5min":1.333,"15min":1.311},"min":{"1min":1.148,"5min":0.802,"15min":0.802},"max":{"1min":2.166,"5min":2.175,"15min":2.175},"last":1.350}]},{"osd":8,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.484,"15min":1.476},"min":{"1min":0.982,"5min":0.982,"15min":0.982},"max":{"1min":2.315,"5min":2.315,"15min":2.315},"last":1.621},{"interface":"front","average":{"1min":1.618,"5min":5.511,"15min":6.160},"min":{"1min":1.196,"5min":1.196,"15min":1.196},"max":{"1min":2.334,"5min":91.314,"15min":91.314},"last":1.548}]},{"osd":9,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":1.204,"5min":1.204,"15min":1.204},"max":{"1min":1.880,"5min":1.880,"15min":1.880},"last":1.511},{"interface":"front","average":{"1min":1.543,"5min":1.543,"15min":1.543},"min":{"1min":1.177,"5min":1.177,"15min":1.177},"max":{"1min":2.430,"5min":2.430,"15min":2.430},"last":184.655}]},{"osd":10,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":1.135,"5min":1.135,"15min":1.135},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":1.475},{"interface":"front","average":{"1min":1.538,"5min":1.538,"15min":1.538},"min":{"1min":1.065,"5min":1.065,"15min":1.065},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.573}]}]}]}} 2019-12-22T22:28:12.877 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2019-12-22T22:28:12.877 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2019-12-22T22:28:12.877 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2019-12-22T22:28:12.878 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph health --format=json 2019-12-22T22:28:15.102 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:28:15.598 INFO:teuthology.orchestra.run.smithi091.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2019-12-22T22:28:15.599 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2019-12-22T22:28:15.599 INFO:tasks.cephadm:Setup complete, yielding 2019-12-22T22:28:15.599 INFO:teuthology.run_tasks:Running task exec... 2019-12-22T22:28:15.635 INFO:teuthology.task.exec:Executing custom commands... 2019-12-22T22:28:15.635 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi091.front.sepia.ceph.com 2019-12-22T22:28:15.636 INFO:teuthology.orchestra.run.smithi091:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'while ! ceph balancer status ; do sleep 1 ; done' 2019-12-22T22:28:16.247 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:28:16.247 INFO:teuthology.orchestra.run.smithi091.stdout: "active": false, 2019-12-22T22:28:16.247 INFO:teuthology.orchestra.run.smithi091.stdout: "last_optimize_duration": "", 2019-12-22T22:28:16.247 INFO:teuthology.orchestra.run.smithi091.stdout: "last_optimize_started": "", 2019-12-22T22:28:16.248 INFO:teuthology.orchestra.run.smithi091.stdout: "mode": "none", 2019-12-22T22:28:16.248 INFO:teuthology.orchestra.run.smithi091.stdout: "optimize_result": "", 2019-12-22T22:28:16.248 INFO:teuthology.orchestra.run.smithi091.stdout: "plans": [] 2019-12-22T22:28:16.248 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:28:16.266 INFO:teuthology.orchestra.run.smithi091:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph balancer mode crush-compat' 2019-12-22T22:28:17.084 INFO:teuthology.orchestra.run.smithi091:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph balancer on' 2019-12-22T22:28:17.689 INFO:teuthology.run_tasks:Running task thrashosds... 2019-12-22T22:28:17.737 INFO:tasks.thrashosds:config is {'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'optrack_toggle_delay': 2.0, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'chance_pgnum_grow': 2, 'sighup_delay': 0.1} 2019-12-22T22:28:17.737 INFO:tasks.thrashosds:overrides is {'ceph': {'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', '\\(MON_DOWN\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow requests'], 'mon_bind_msgr2': False, 'conf': {'client.2': {'debug objecter': 20, 'debug ms': 1, 'debug rados': 20}, 'global': {'ms inject delay max': 1, 'ms inject delay type': 'osd', 'osd_pool_default_min_size': 2, 'mon warn on msgr2 not enabled': False, 'ms inject delay probability': 0.005, 'mon client directed command retry': 5, 'ms inject socket failures': 2500, 'osd_pool_default_size': 2, 'ms inject internal delays': 0.002}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'debug ms': 1, 'mon osdmap full prune min': 15, 'mon osdmap full prune interval': 2, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'osd op queue cut off': 'debug_random', 'debug ms': 1, 'debug journal': 20, 'osd scrub max interval': 120, 'osd max backfills': 2, 'debug osd': 25, 'osd snap trim sleep': 0.5, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'filestore odsync write': True, 'osd op queue': 'debug_random', 'osd backoff on peering': True, 'osd scrub min interval': 60, 'debug filestore': 20, 'osd shutdown pgref assert': True}}, 'sha1': '99b509ffd4dd354919b08b38ba83b998fbfdc107'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'workunit': {'sha1': 'abe4c1c94873149a777c88f919a579d57de0f422', 'branch': 'wip-old-clients-cephadm'}, 'install': {'ceph': {'sha1': '99b509ffd4dd354919b08b38ba83b998fbfdc107'}}, 'admin_socket': {'branch': 'wip-sage-testing-2019-12-22-0931'}} 2019-12-22T22:28:17.738 INFO:tasks.thrashosds:config is {'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'optrack_toggle_delay': 2.0, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'chance_pgnum_grow': 2, 'sighup_delay': 0.1} 2019-12-22T22:28:17.738 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2019-12-22T22:28:20.472 INFO:tasks.thrashosds:Beginning thrashosds... 2019-12-22T22:28:20.473 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2019-12-22T22:28:20.473 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph pg dump --format=json 2019-12-22T22:28:22.738 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:28:22.739 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2019-12-22T22:28:23.293 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2019-12-22T22:28:20.917879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":12,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1124941824,"kb_used":12590476,"kb_used_data":7148,"kb_used_omap":0,"kb_used_meta":12582912,"kb_avail":1112351348,"statfs":{"total":1151940427776,"available":1139047780352,"internally_reserved":12884901888,"allocated":7319552,"data_stored":3428479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12884901888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[],"pool_stats":[],"osd_stats":[{"osd":11,"up_from":0,"seq":19,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.618,"5min":0.618,"15min":0.618},"max":{"1min":2.561,"5min":2.561,"15min":2.561},"last":0.593},{"interface":"front","average":{"1min":0.962,"5min":0.962,"15min":0.962},"min":{"1min":0.641,"5min":0.641,"15min":0.641},"max":{"1min":1.942,"5min":1.942,"15min":1.942},"last":0.783}]},{"osd":1,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":0.856,"5min":0.856,"15min":0.856},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.936},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.707,"5min":0.707,"15min":0.707},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":1.111}]},{"osd":2,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.037,"5min":1.037,"15min":1.037},"min":{"1min":0.812,"5min":0.812,"15min":0.812},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.053},{"interface":"front","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.656,"5min":0.656,"15min":0.656},"max":{"1min":1.312,"5min":1.312,"15min":1.312},"last":1.183}]},{"osd":3,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.687,"5min":0.687,"15min":0.687},"max":{"1min":1.666,"5min":1.666,"15min":1.666},"last":0.863},{"interface":"front","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.692,"5min":0.692,"15min":0.692},"max":{"1min":2.688,"5min":2.688,"15min":2.688},"last":0.993}]},{"osd":4,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.533,"5min":1.533,"15min":1.533},"min":{"1min":0.955,"5min":0.955,"15min":0.955},"max":{"1min":3.629,"5min":3.629,"15min":3.629},"last":1.310},{"interface":"front","average":{"1min":1.270,"5min":1.270,"15min":1.270},"min":{"1min":0.761,"5min":0.761,"15min":0.761},"max":{"1min":1.894,"5min":1.894,"15min":1.894},"last":1.425}]},{"osd":5,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.820,"5min":0.820,"15min":0.820},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.358},{"interface":"front","average":{"1min":1.686,"5min":1.686,"15min":1.686},"min":{"1min":1.128,"5min":1.128,"15min":1.128},"max":{"1min":3.739,"5min":3.739,"15min":3.739},"last":1.497}]},{"osd":6,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":35.379,"5min":35.379,"15min":35.379},"min":{"1min":0.984,"5min":0.984,"15min":0.984},"max":{"1min":646.716,"5min":646.716,"15min":646.716},"last":1.262},{"interface":"front","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.905,"5min":0.905,"15min":0.905},"max":{"1min":2.199,"5min":2.199,"15min":2.199},"last":1.554}]},{"osd":7,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.558,"5min":1.558,"15min":1.558},"min":{"1min":1.116,"5min":1.116,"15min":1.116},"max":{"1min":2.283,"5min":2.283,"15min":2.283},"last":1.619},{"interface":"front","average":{"1min":1.566,"5min":1.566,"15min":1.566},"min":{"1min":1.204,"5min":1.204,"15min":1.204},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":1.711}]},{"osd":8,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.822,"5min":1.822,"15min":1.822},"min":{"1min":1.237,"5min":1.237,"15min":1.237},"max":{"1min":3.936,"5min":3.936,"15min":3.936},"last":1.826},{"interface":"front","average":{"1min":34.440,"5min":34.440,"15min":34.440},"min":{"1min":1.332,"5min":1.332,"15min":1.332},"max":{"1min":624.904,"5min":624.904,"15min":624.904},"last":1.886}]},{"osd":10,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.871,"5min":1.871,"15min":1.871},"min":{"1min":1.205,"5min":1.205,"15min":1.205},"max":{"1min":4.041,"5min":4.041,"15min":4.041},"last":1.661},{"interface":"front","average":{"1min":1.772,"5min":1.772,"15min":1.772},"min":{"1min":1.354,"5min":1.354,"15min":1.354},"max":{"1min":3.823,"5min":3.823,"15min":3.823},"last":1.782}]}]},{"osd":10,"up_from":0,"seq":22,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":1.404}]},{"osd":1,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.667,"5min":0.667,"15min":0.667},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.512},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.712,"5min":0.712,"15min":0.712},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":1.368}]},{"osd":2,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.549,"5min":0.549,"15min":0.549},"max":{"1min":1.685,"5min":1.685,"15min":1.685},"last":1.870},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.604,"5min":0.604,"15min":0.604},"max":{"1min":2.191,"5min":2.191,"15min":2.191},"last":1.630}]},{"osd":3,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.010,"5min":1.010,"15min":1.010},"min":{"1min":0.635,"5min":0.635,"15min":0.635},"max":{"1min":1.438,"5min":1.438,"15min":1.438},"last":1.711},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.883,"5min":0.883,"15min":0.883},"max":{"1min":1.646,"5min":1.646,"15min":1.646},"last":1.667}]},{"osd":4,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.588,"5min":1.588,"15min":1.588},"min":{"1min":1.003,"5min":1.003,"15min":1.003},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":1.197},{"interface":"front","average":{"1min":1.915,"5min":1.915,"15min":1.915},"min":{"1min":0.740,"5min":0.740,"15min":0.740},"max":{"1min":11.454,"5min":11.454,"15min":11.454},"last":1.564}]},{"osd":5,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.430,"5min":1.430,"15min":1.430},"min":{"1min":0.757,"5min":0.757,"15min":0.757},"max":{"1min":2.450,"5min":2.450,"15min":2.450},"last":0.773},{"interface":"front","average":{"1min":1.681,"5min":1.681,"15min":1.681},"min":{"1min":1.204,"5min":1.204,"15min":1.204},"max":{"1min":2.777,"5min":2.777,"15min":2.777},"last":0.866}]},{"osd":6,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":2.012,"5min":2.012,"15min":2.012},"min":{"1min":1.031,"5min":1.031,"15min":1.031},"max":{"1min":11.404,"5min":11.404,"15min":11.404},"last":1.042},{"interface":"front","average":{"1min":1.559,"5min":1.559,"15min":1.559},"min":{"1min":1.108,"5min":1.108,"15min":1.108},"max":{"1min":2.462,"5min":2.462,"15min":2.462},"last":0.934}]},{"osd":7,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":102.420,"5min":102.420,"15min":102.420},"min":{"1min":1.064,"5min":1.064,"15min":1.064},"max":{"1min":938.067,"5min":938.067,"15min":938.067},"last":1.241},{"interface":"front","average":{"1min":1.644,"5min":1.644,"15min":1.644},"min":{"1min":1.151,"5min":1.151,"15min":1.151},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":1.135}]},{"osd":8,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":2.393,"5min":2.393,"15min":2.393},"min":{"1min":1.068,"5min":1.068,"15min":1.068},"max":{"1min":11.238,"5min":11.238,"15min":11.238},"last":1.771},{"interface":"front","average":{"1min":2.286,"5min":2.286,"15min":2.286},"min":{"1min":0.928,"5min":0.928,"15min":0.928},"max":{"1min":11.482,"5min":11.482,"15min":11.482},"last":1.814}]},{"osd":9,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":2.448,"5min":2.448,"15min":2.448},"min":{"1min":1.242,"5min":1.242,"15min":1.242},"max":{"1min":11.328,"5min":11.328,"15min":11.328},"last":806.388},{"interface":"front","average":{"1min":2.308,"5min":2.308,"15min":2.308},"min":{"1min":1.182,"5min":1.182,"15min":1.182},"max":{"1min":11.191,"5min":11.191,"15min":11.191},"last":1.407}]},{"osd":11,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.723,"5min":1.723,"15min":1.723},"min":{"1min":1.306,"5min":1.306,"15min":1.306},"max":{"1min":2.336,"5min":2.336,"15min":2.336},"last":1.286},{"interface":"front","average":{"1min":1.915,"5min":1.915,"15min":1.915},"min":{"1min":1.594,"5min":1.594,"15min":1.594},"max":{"1min":2.620,"5min":2.620,"15min":2.620},"last":1.466}]}]},{"osd":9,"up_from":0,"seq":27,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.084,"5min":1.084,"15min":1.084},"min":{"1min":0.636,"5min":0.636,"15min":0.636},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.208},{"interface":"front","average":{"1min":0.973,"5min":0.973,"15min":0.973},"min":{"1min":0.604,"5min":0.604,"15min":0.604},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":1.073}]},{"osd":1,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.037,"5min":1.037,"15min":1.037},"min":{"1min":0.656,"5min":0.656,"15min":0.656},"max":{"1min":1.664,"5min":1.664,"15min":1.664},"last":1.258},{"interface":"front","average":{"1min":0.891,"5min":0.891,"15min":0.891},"min":{"1min":0.647,"5min":0.647,"15min":0.647},"max":{"1min":1.437,"5min":1.437,"15min":1.437},"last":0.882}]},{"osd":2,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.197,"5min":1.197,"15min":1.197},"min":{"1min":0.738,"5min":0.738,"15min":0.738},"max":{"1min":2.162,"5min":2.162,"15min":2.162},"last":1.101},{"interface":"front","average":{"1min":27.186,"5min":27.186,"15min":27.186},"min":{"1min":0.668,"5min":0.668,"15min":0.668},"max":{"1min":524.094,"5min":524.094,"15min":524.094},"last":0.703}]},{"osd":3,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":35.175,"5min":35.175,"15min":35.175},"min":{"1min":0.538,"5min":0.538,"15min":0.538},"max":{"1min":683.851,"5min":683.851,"15min":683.851},"last":1.167},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.599,"5min":0.599,"15min":0.599},"max":{"1min":2.067,"5min":2.067,"15min":2.067},"last":1.342}]},{"osd":4,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.219,"5min":1.219,"15min":1.219},"min":{"1min":0.840,"5min":0.840,"15min":0.840},"max":{"1min":1.574,"5min":1.574,"15min":1.574},"last":0.990},{"interface":"front","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.716,"5min":0.716,"15min":0.716},"max":{"1min":1.720,"5min":1.720,"15min":1.720},"last":1.299}]},{"osd":5,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.275,"5min":1.275,"15min":1.275},"min":{"1min":0.693,"5min":0.693,"15min":0.693},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.440},{"interface":"front","average":{"1min":1.533,"5min":1.533,"15min":1.533},"min":{"1min":1.070,"5min":1.070,"15min":1.070},"max":{"1min":2.252,"5min":2.252,"15min":2.252},"last":1.591}]},{"osd":6,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":14.726,"5min":14.726,"15min":14.726},"min":{"1min":1.131,"5min":1.131,"15min":1.131},"max":{"1min":264.274,"5min":264.274,"15min":264.274},"last":1.536},{"interface":"front","average":{"1min":1.342,"5min":1.342,"15min":1.342},"min":{"1min":1.016,"5min":1.016,"15min":1.016},"max":{"1min":1.660,"5min":1.660,"15min":1.660},"last":1.020}]},{"osd":7,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.453,"5min":1.453,"15min":1.453},"min":{"1min":0.757,"5min":0.757,"15min":0.757},"max":{"1min":1.965,"5min":1.965,"15min":1.965},"last":1.404},{"interface":"front","average":{"1min":1.697,"5min":1.697,"15min":1.697},"min":{"1min":1.248,"5min":1.248,"15min":1.248},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":1.375}]},{"osd":8,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.443,"5min":1.443,"15min":1.443},"min":{"1min":0.700,"5min":0.700,"15min":0.700},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.049},{"interface":"front","average":{"1min":1.567,"5min":1.567,"15min":1.567},"min":{"1min":0.919,"5min":0.919,"15min":0.919},"max":{"1min":2.023,"5min":2.023,"15min":2.023},"last":1.476}]},{"osd":10,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.718,"5min":1.718,"15min":1.718},"min":{"1min":1.381,"5min":1.381,"15min":1.381},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":1.513},{"interface":"front","average":{"1min":1.802,"5min":1.802,"15min":1.802},"min":{"1min":1.407,"5min":1.407,"15min":1.407},"max":{"1min":2.188,"5min":2.188,"15min":2.188},"last":1.638}]}]},{"osd":8,"up_from":0,"seq":31,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049212,"kb_used_data":548,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695940,"statfs":{"total":95995035648,"available":94920642560,"internally_reserved":1073741824,"allocated":561152,"data_stored":269721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.122,"5min":1.090,"15min":1.084},"min":{"1min":0.631,"5min":0.551,"15min":0.551},"max":{"1min":4.025,"5min":4.025,"15min":4.025},"last":0.901},{"interface":"front","average":{"1min":1.276,"5min":25.673,"15min":29.739},"min":{"1min":0.624,"5min":0.507,"15min":0.507},"max":{"1min":3.963,"5min":586.314,"15min":586.314},"last":1.392}]},{"osd":1,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":16.711,"5min":4.286,"15min":2.215},"min":{"1min":0.688,"5min":0.593,"15min":0.593},"max":{"1min":312.656,"5min":312.656,"15min":312.656},"last":0.850},{"interface":"front","average":{"1min":1.201,"5min":1.077,"15min":1.057},"min":{"1min":0.674,"5min":0.589,"15min":0.589},"max":{"1min":4.160,"5min":4.160,"15min":4.160},"last":0.732}]},{"osd":2,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.289,"5min":1.222,"15min":1.211},"min":{"1min":0.513,"5min":0.513,"15min":0.513},"max":{"1min":3.922,"5min":3.922,"15min":3.922},"last":1.002},{"interface":"front","average":{"1min":1.393,"5min":1.299,"15min":1.283},"min":{"1min":0.873,"5min":0.653,"15min":0.653},"max":{"1min":4.084,"5min":4.084,"15min":4.084},"last":0.794}]},{"osd":3,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.304,"15min":1.310},"min":{"1min":0.582,"5min":0.582,"15min":0.582},"max":{"1min":4.349,"5min":4.349,"15min":4.349},"last":0.941},{"interface":"front","average":{"1min":1.022,"5min":29.273,"15min":33.981},"min":{"1min":0.579,"5min":0.579,"15min":0.579},"max":{"1min":4.267,"5min":667.228,"15min":667.228},"last":1.326}]},{"osd":4,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.132,"5min":1.352,"15min":1.388},"min":{"1min":0.790,"5min":0.717,"15min":0.717},"max":{"1min":1.520,"5min":2.569,"15min":2.569},"last":1.096},{"interface":"front","average":{"1min":1.449,"5min":16.140,"15min":18.588},"min":{"1min":1.056,"5min":0.805,"15min":0.805},"max":{"1min":1.838,"5min":365.920,"15min":365.920},"last":1.452}]},{"osd":5,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.480,"15min":1.524},"min":{"1min":0.883,"5min":0.883,"15min":0.883},"max":{"1min":1.784,"5min":2.683,"15min":2.683},"last":1.269},{"interface":"front","average":{"1min":19.072,"5min":5.025,"15min":2.684},"min":{"1min":0.683,"5min":0.683,"15min":0.683},"max":{"1min":356.189,"5min":356.189,"15min":356.189},"last":1.134}]},{"osd":6,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.630,"15min":1.646},"min":{"1min":1.289,"5min":0.945,"15min":0.945},"max":{"1min":1.926,"5min":2.406,"15min":2.406},"last":1.819},{"interface":"front","average":{"1min":1.608,"5min":1.629,"15min":1.633},"min":{"1min":0.830,"5min":0.830,"15min":0.830},"max":{"1min":1.960,"5min":2.715,"15min":2.715},"last":1.506}]},{"osd":7,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":12.654,"15min":14.559},"min":{"1min":0.842,"5min":0.570,"15min":0.570},"max":{"1min":1.623,"5min":282.817,"15min":282.817},"last":1.060},{"interface":"front","average":{"1min":1.645,"5min":1.688,"15min":1.695},"min":{"1min":1.338,"5min":0.806,"15min":0.806},"max":{"1min":1.982,"5min":2.525,"15min":2.525},"last":1.736}]},{"osd":9,"last update":"Sun Dec 22 22:27:16 2019","interfaces":[{"interface":"back","average":{"1min":52.511,"5min":52.511,"15min":52.511},"min":{"1min":0.525,"5min":0.525,"15min":0.525},"max":{"1min":686.274,"5min":686.274,"15min":686.274},"last":1.640},{"interface":"front","average":{"1min":1.675,"5min":1.675,"15min":1.675},"min":{"1min":0.464,"5min":0.464,"15min":0.464},"max":{"1min":2.587,"5min":2.587,"15min":2.587},"last":1.686}]},{"osd":10,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":19.663,"5min":19.663,"15min":19.663},"min":{"1min":0.732,"5min":0.732,"15min":0.732},"max":{"1min":358.731,"5min":358.731,"15min":358.731},"last":1.539},{"interface":"front","average":{"1min":1.829,"5min":1.829,"15min":1.829},"min":{"1min":0.843,"5min":0.843,"15min":0.843},"max":{"1min":2.791,"5min":2.791,"15min":2.791},"last":1.580}]}]},{"osd":7,"up_from":0,"seq":34,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":0.973,"5min":0.854,"15min":0.834},"min":{"1min":0.558,"5min":0.558,"15min":0.558},"max":{"1min":1.443,"5min":1.443,"15min":1.443},"last":0.999},{"interface":"front","average":{"1min":14.982,"5min":3.818,"15min":1.958},"min":{"1min":0.731,"5min":0.684,"15min":0.684},"max":{"1min":291.908,"5min":291.908,"15min":291.908},"last":0.807}]},{"osd":1,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.197,"15min":1.201},"min":{"1min":0.819,"5min":0.758,"15min":0.758},"max":{"1min":1.616,"5min":2.308,"15min":2.308},"last":0.734},{"interface":"front","average":{"1min":1.079,"5min":1.059,"15min":1.056},"min":{"1min":0.680,"5min":0.680,"15min":0.680},"max":{"1min":1.396,"5min":1.937,"15min":1.937},"last":1.131}]},{"osd":2,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":0.991,"5min":1.018,"15min":1.022},"min":{"1min":0.685,"5min":0.685,"15min":0.685},"max":{"1min":1.529,"5min":1.886,"15min":1.886},"last":0.899},{"interface":"front","average":{"1min":1.121,"5min":1.145,"15min":1.149},"min":{"1min":0.817,"5min":0.625,"15min":0.625},"max":{"1min":1.524,"5min":2.376,"15min":2.376},"last":0.766}]},{"osd":3,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":42.500,"15min":49.405},"min":{"1min":0.648,"5min":0.648,"15min":0.648},"max":{"1min":1.494,"5min":985.876,"15min":985.876},"last":1.393},{"interface":"front","average":{"1min":1.217,"5min":1.272,"15min":1.281},"min":{"1min":0.784,"5min":0.694,"15min":0.694},"max":{"1min":1.559,"5min":2.153,"15min":2.153},"last":0.850}]},{"osd":4,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.143,"15min":1.151},"min":{"1min":0.699,"5min":0.600,"15min":0.600},"max":{"1min":1.654,"5min":1.800,"15min":1.800},"last":1.236},{"interface":"front","average":{"1min":1.159,"5min":1.191,"15min":1.196},"min":{"1min":0.704,"5min":0.704,"15min":0.704},"max":{"1min":1.643,"5min":1.833,"15min":1.833},"last":1.169}]},{"osd":5,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.154,"5min":1.273,"15min":1.293},"min":{"1min":0.633,"5min":0.633,"15min":0.633},"max":{"1min":1.629,"5min":1.663,"15min":1.663},"last":1.358},{"interface":"front","average":{"1min":1.398,"5min":1.387,"15min":1.385},"min":{"1min":0.855,"5min":0.728,"15min":0.728},"max":{"1min":1.747,"5min":2.435,"15min":2.435},"last":1.090}]},{"osd":6,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.322,"5min":1.273,"15min":1.265},"min":{"1min":0.611,"5min":0.611,"15min":0.611},"max":{"1min":1.710,"5min":2.106,"15min":2.106},"last":1.281},{"interface":"front","average":{"1min":1.369,"5min":31.793,"15min":36.864},"min":{"1min":0.976,"5min":0.976,"15min":0.976},"max":{"1min":1.755,"5min":684.801,"15min":684.801},"last":1.475}]},{"osd":8,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":29.540,"5min":7.085,"15min":3.343},"min":{"1min":0.901,"5min":0.807,"15min":0.807},"max":{"1min":502.217,"5min":502.217,"15min":502.217},"last":1.440},{"interface":"front","average":{"1min":1.560,"5min":1.628,"15min":1.639},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":1.788,"5min":2.619,"15min":2.619},"last":1.210}]},{"osd":9,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.606,"5min":1.606,"15min":1.606},"min":{"1min":1.228,"5min":1.228,"15min":1.228},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.579},{"interface":"front","average":{"1min":46.020,"5min":46.020,"15min":46.020},"min":{"1min":1.173,"5min":1.173,"15min":1.173},"max":{"1min":891.130,"5min":891.130,"15min":891.130},"last":1.505}]},{"osd":10,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":1.671,"15min":1.671},"min":{"1min":1.193,"5min":1.193,"15min":1.193},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.329},{"interface":"front","average":{"1min":53.065,"5min":53.065,"15min":53.065},"min":{"1min":1.327,"5min":1.327,"15min":1.327},"max":{"1min":748.773,"5min":748.773,"15min":748.773},"last":1.531}]}]},{"osd":6,"up_from":0,"seq":40,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":1.053,"15min":1.083},"min":{"1min":0.601,"5min":0.601,"15min":0.601},"max":{"1min":1.682,"5min":1.682,"15min":1.682},"last":1.386},{"interface":"front","average":{"1min":0.974,"5min":1.074,"15min":1.090},"min":{"1min":0.689,"5min":0.689,"15min":0.689},"max":{"1min":1.528,"5min":1.677,"15min":1.677},"last":1.148}]},{"osd":1,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.893,"15min":0.895},"min":{"1min":0.626,"5min":0.591,"15min":0.591},"max":{"1min":1.364,"5min":1.793,"15min":1.793},"last":0.602},{"interface":"front","average":{"1min":0.920,"5min":19.041,"15min":22.061},"min":{"1min":0.611,"5min":0.588,"15min":0.588},"max":{"1min":1.628,"5min":363.766,"15min":363.766},"last":0.887}]},{"osd":2,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":5.483,"5min":1.961,"15min":1.374},"min":{"1min":0.674,"5min":0.674,"15min":0.674},"max":{"1min":89.000,"5min":89.000,"15min":89.000},"last":105.857},{"interface":"front","average":{"1min":1.033,"5min":46.178,"15min":53.702},"min":{"1min":0.721,"5min":0.721,"15min":0.721},"max":{"1min":1.550,"5min":904.141,"15min":904.141},"last":0.733}]},{"osd":3,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":16.675,"15min":19.261},"min":{"1min":0.902,"5min":0.834,"15min":0.834},"max":{"1min":1.511,"5min":312.765,"15min":312.765},"last":0.766},{"interface":"front","average":{"1min":1.160,"5min":1.184,"15min":1.188},"min":{"1min":0.719,"5min":0.719,"15min":0.719},"max":{"1min":1.468,"5min":1.771,"15min":1.771},"last":0.809}]},{"osd":4,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":47.470,"5min":10.303,"15min":4.109},"min":{"1min":0.727,"5min":0.535,"15min":0.535},"max":{"1min":882.404,"5min":882.404,"15min":882.404},"last":1.015},{"interface":"front","average":{"1min":1.110,"5min":1.036,"15min":1.024},"min":{"1min":0.653,"5min":0.595,"15min":0.595},"max":{"1min":1.572,"5min":1.572,"15min":1.572},"last":0.970}]},{"osd":5,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.197,"5min":1.164,"15min":1.158},"min":{"1min":0.843,"5min":0.680,"15min":0.680},"max":{"1min":1.453,"5min":1.839,"15min":1.839},"last":1.247},{"interface":"front","average":{"1min":1.296,"5min":1.257,"15min":1.251},"min":{"1min":0.960,"5min":0.896,"15min":0.896},"max":{"1min":1.706,"5min":1.804,"15min":1.804},"last":1.268}]},{"osd":7,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.311,"15min":1.307},"min":{"1min":1.073,"5min":1.030,"15min":1.030},"max":{"1min":1.590,"5min":1.675,"15min":1.675},"last":1.184},{"interface":"front","average":{"1min":1.392,"5min":1.350,"15min":1.343},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.656,"5min":1.721,"15min":1.721},"last":1.470}]},{"osd":8,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":49.165,"15min":57.102},"min":{"1min":1.168,"5min":1.168,"15min":1.168},"max":{"1min":1.972,"5min":844.789,"15min":844.789},"last":1.613},{"interface":"front","average":{"1min":1.408,"5min":1.431,"15min":1.435},"min":{"1min":1.200,"5min":1.183,"15min":1.183},"max":{"1min":1.840,"5min":1.840,"15min":1.840},"last":1.537}]},{"osd":9,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.531,"5min":1.531,"15min":1.531},"min":{"1min":1.113,"5min":1.113,"15min":1.113},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":1.650},{"interface":"front","average":{"1min":1.485,"5min":1.485,"15min":1.485},"min":{"1min":1.043,"5min":1.043,"15min":1.043},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.426}]},{"osd":10,"last update":"Sun Dec 22 22:27:37 2019","interfaces":[{"interface":"back","average":{"1min":1.497,"5min":1.497,"15min":1.497},"min":{"1min":1.156,"5min":1.156,"15min":1.156},"max":{"1min":1.866,"5min":1.866,"15min":1.866},"last":1.571},{"interface":"front","average":{"1min":1.596,"5min":1.596,"15min":1.596},"min":{"1min":1.000,"5min":1.000,"15min":1.000},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.713}]}]},{"osd":1,"up_from":0,"seq":60,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049236,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695916,"statfs":{"total":95995035648,"available":94920617984,"internally_reserved":1073741824,"allocated":585728,"data_stored":284195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":0.805,"5min":0.645,"15min":0.595},"min":{"1min":0.525,"5min":0.338,"15min":0.338},"max":{"1min":1.279,"5min":1.279,"15min":1.279},"last":0.827},{"interface":"front","average":{"1min":0.901,"5min":0.829,"15min":0.735},"min":{"1min":0.507,"5min":0.381,"15min":0.381},"max":{"1min":1.370,"5min":1.511,"15min":1.511},"last":1.506}]},{"osd":2,"last update":"Sun Dec 22 22:28:05 2019","interfaces":[{"interface":"back","average":{"1min":1.011,"5min":0.885,"15min":0.797},"min":{"1min":0.500,"5min":0.391,"15min":0.391},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.631},{"interface":"front","average":{"1min":0.869,"5min":3.282,"15min":1.544},"min":{"1min":0.475,"5min":0.436,"15min":0.436},"max":{"1min":1.471,"5min":254.847,"15min":254.847},"last":0.979}]},{"osd":3,"last update":"Sun Dec 22 22:28:18 2019","interfaces":[{"interface":"back","average":{"1min":1.068,"5min":18.341,"15min":6.645},"min":{"1min":0.654,"5min":0.551,"15min":0.551},"max":{"1min":2.025,"5min":952.051,"15min":952.051},"last":1.382},{"interface":"front","average":{"1min":1.174,"5min":15.304,"15min":13.613},"min":{"1min":0.700,"5min":0.620,"15min":0.620},"max":{"1min":2.202,"5min":903.487,"15min":903.487},"last":1.756}]},{"osd":4,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":1.010,"5min":0.878,"15min":0.847},"min":{"1min":0.707,"5min":0.601,"15min":0.601},"max":{"1min":1.424,"5min":1.424,"15min":1.424},"last":1.131},{"interface":"front","average":{"1min":1.338,"5min":1.193,"15min":1.117},"min":{"1min":0.946,"5min":0.732,"15min":0.732},"max":{"1min":1.685,"5min":1.941,"15min":1.941},"last":1.694}]},{"osd":5,"last update":"Sun Dec 22 22:28:02 2019","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":0.977,"15min":0.957},"min":{"1min":0.851,"5min":0.676,"15min":0.676},"max":{"1min":1.521,"5min":1.521,"15min":1.521},"last":1.085},{"interface":"front","average":{"1min":12.769,"5min":3.568,"15min":2.000},"min":{"1min":0.809,"5min":0.809,"15min":0.809},"max":{"1min":263.068,"5min":263.068,"15min":263.068},"last":1.873}]},{"osd":6,"last update":"Sun Dec 22 22:27:23 2019","interfaces":[{"interface":"back","average":{"1min":1.410,"5min":1.411,"15min":1.411},"min":{"1min":0.972,"5min":0.972,"15min":0.972},"max":{"1min":2.000,"5min":2.315,"15min":2.315},"last":1.819},{"interface":"front","average":{"1min":1.019,"5min":1.013,"15min":1.012},"min":{"1min":0.736,"5min":0.736,"15min":0.736},"max":{"1min":1.527,"5min":1.673,"15min":1.673},"last":1.334}]},{"osd":7,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":1.400,"5min":1.473,"15min":1.485},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.861,"5min":2.238,"15min":2.238},"last":2.000},{"interface":"front","average":{"1min":6.449,"5min":2.532,"15min":1.879},"min":{"1min":0.963,"5min":0.963,"15min":0.963},"max":{"1min":101.788,"5min":101.788,"15min":101.788},"last":2.067}]},{"osd":8,"last update":"Sun Dec 22 22:28:02 2019","interfaces":[{"interface":"back","average":{"1min":1.189,"5min":1.189,"15min":1.189},"min":{"1min":0.826,"5min":0.826,"15min":0.826},"max":{"1min":1.594,"5min":1.594,"15min":1.594},"last":1.283},{"interface":"front","average":{"1min":1.196,"5min":1.228,"15min":1.233},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":1.566}]},{"osd":9,"last update":"Sun Dec 22 22:28:18 2019","interfaces":[{"interface":"back","average":{"1min":1.594,"5min":1.590,"15min":1.589},"min":{"1min":0.984,"5min":0.984,"15min":0.984},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":2.184},{"interface":"front","average":{"1min":1.259,"5min":12.659,"15min":14.559},"min":{"1min":0.595,"5min":0.595,"15min":0.595},"max":{"1min":1.593,"5min":300.557,"15min":300.557},"last":1.459}]},{"osd":10,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":46.030,"5min":46.030,"15min":46.030},"min":{"1min":1.021,"5min":1.021,"15min":1.021},"max":{"1min":896.161,"5min":896.161,"15min":896.161},"last":1.228},{"interface":"front","average":{"1min":1.528,"5min":1.528,"15min":1.528},"min":{"1min":1.144,"5min":1.144,"15min":1.144},"max":{"1min":1.902,"5min":1.902,"15min":1.902},"last":2.128}]}]},{"osd":0,"up_from":0,"seq":64,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":0.912,"5min":0.792,"15min":0.752},"min":{"1min":0.582,"5min":0.291,"15min":0.291},"max":{"1min":1.381,"5min":1.720,"15min":1.720},"last":0.657},{"interface":"front","average":{"1min":1.068,"5min":0.864,"15min":0.760},"min":{"1min":0.659,"5min":0.349,"15min":0.349},"max":{"1min":1.570,"5min":1.597,"15min":1.597},"last":0.779}]},{"osd":2,"last update":"Sun Dec 22 22:26:03 2019","interfaces":[{"interface":"back","average":{"1min":0.938,"5min":0.922,"15min":0.919},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.347,"5min":1.798,"15min":1.798},"last":1.188},{"interface":"front","average":{"1min":38.337,"5min":38.242,"15min":38.226},"min":{"1min":0.580,"5min":0.552,"15min":0.552},"max":{"1min":601.565,"5min":601.565,"15min":601.565},"last":0.645}]},{"osd":3,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.047,"5min":0.844,"15min":0.776},"min":{"1min":0.616,"5min":0.540,"15min":0.540},"max":{"1min":1.722,"5min":1.832,"15min":1.832},"last":0.718},{"interface":"front","average":{"1min":0.724,"5min":12.273,"15min":4.552},"min":{"1min":0.525,"5min":0.525,"15min":0.525},"max":{"1min":1.155,"5min":867.546,"15min":867.546},"last":0.560}]},{"osd":4,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":0.927,"5min":0.888,"15min":0.848},"min":{"1min":0.673,"5min":0.664,"15min":0.664},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.903},{"interface":"front","average":{"1min":1.313,"5min":1.149,"15min":1.077},"min":{"1min":0.945,"5min":0.850,"15min":0.850},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.924}]},{"osd":5,"last update":"Sun Dec 22 22:28:04 2019","interfaces":[{"interface":"back","average":{"1min":1.273,"5min":1.207,"15min":1.156},"min":{"1min":0.875,"5min":0.802,"15min":0.802},"max":{"1min":1.922,"5min":1.922,"15min":1.922},"last":1.200},{"interface":"front","average":{"1min":0.907,"5min":0.982,"15min":0.970},"min":{"1min":0.654,"5min":0.654,"15min":0.654},"max":{"1min":1.647,"5min":1.887,"15min":1.887},"last":0.826}]},{"osd":6,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.073,"15min":1.070},"min":{"1min":0.709,"5min":0.709,"15min":0.709},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.860},{"interface":"front","average":{"1min":1.480,"5min":26.547,"15min":30.725},"min":{"1min":0.923,"5min":0.923,"15min":0.923},"max":{"1min":2.246,"5min":504.570,"15min":504.570},"last":1.125}]},{"osd":7,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.195,"15min":1.199},"min":{"1min":0.863,"5min":0.863,"15min":0.863},"max":{"1min":1.616,"5min":1.616,"15min":1.616},"last":0.958},{"interface":"front","average":{"1min":1.488,"5min":26.395,"15min":30.546},"min":{"1min":0.975,"5min":0.975,"15min":0.975},"max":{"1min":1.885,"5min":530.298,"15min":530.298},"last":1.239}]},{"osd":8,"last update":"Sun Dec 22 22:28:04 2019","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.303,"15min":1.319},"min":{"1min":0.815,"5min":0.805,"15min":0.805},"max":{"1min":1.771,"5min":1.771,"15min":1.771},"last":0.980},{"interface":"front","average":{"1min":46.831,"5min":10.652,"15min":4.622},"min":{"1min":0.892,"5min":0.892,"15min":0.892},"max":{"1min":817.964,"5min":817.964,"15min":817.964},"last":1.271}]},{"osd":9,"last update":"Sun Dec 22 22:27:17 2019","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.649,"15min":1.649},"min":{"1min":1.280,"5min":1.280,"15min":1.280},"max":{"1min":2.180,"5min":2.180,"15min":2.180},"last":1.388},{"interface":"front","average":{"1min":1.264,"5min":1.264,"15min":1.264},"min":{"1min":0.969,"5min":0.969,"15min":0.969},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.080}]},{"osd":10,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.639,"15min":1.639},"min":{"1min":1.130,"5min":1.130,"15min":1.130},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":1.323},{"interface":"front","average":{"1min":1.644,"5min":1.644,"15min":1.644},"min":{"1min":1.072,"5min":1.072,"15min":1.072},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":1.355}]},{"osd":11,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.160,"5min":1.160,"15min":1.160},"min":{"1min":0.772,"5min":0.772,"15min":0.772},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.154},{"interface":"front","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":0.853,"5min":0.853,"15min":0.853},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.047}]}]},{"osd":2,"up_from":0,"seq":57,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049156,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695996,"statfs":{"total":95995035648,"available":94920699904,"internally_reserved":1073741824,"allocated":585728,"data_stored":284195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:28:07 2019","interfaces":[{"interface":"back","average":{"1min":0.733,"5min":0.657,"15min":0.567},"min":{"1min":0.418,"5min":0.398,"15min":0.398},"max":{"1min":1.422,"5min":1.461,"15min":1.461},"last":0.698},{"interface":"front","average":{"1min":0.832,"5min":0.699,"15min":0.623},"min":{"1min":0.423,"5min":0.423,"15min":0.423},"max":{"1min":1.684,"5min":1.684,"15min":1.684},"last":0.891}]},{"osd":1,"last update":"Sun Dec 22 22:28:07 2019","interfaces":[{"interface":"back","average":{"1min":0.909,"5min":0.819,"15min":0.745},"min":{"1min":0.586,"5min":0.501,"15min":0.501},"max":{"1min":1.324,"5min":1.555,"15min":1.555},"last":0.753},{"interface":"front","average":{"1min":1.038,"5min":0.810,"15min":0.744},"min":{"1min":0.656,"5min":0.478,"15min":0.478},"max":{"1min":1.406,"5min":1.406,"15min":1.406},"last":0.821}]},{"osd":3,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.029,"5min":0.868,"15min":0.818},"min":{"1min":0.711,"5min":0.563,"15min":0.563},"max":{"1min":1.261,"5min":1.261,"15min":1.261},"last":1.085},{"interface":"front","average":{"1min":0.952,"5min":0.840,"15min":0.818},"min":{"1min":0.611,"5min":0.525,"15min":0.525},"max":{"1min":1.930,"5min":1.930,"15min":1.930},"last":0.767}]},{"osd":4,"last update":"Sun Dec 22 22:27:42 2019","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.019,"15min":0.995},"min":{"1min":0.594,"5min":0.594,"15min":0.594},"max":{"1min":1.947,"5min":1.947,"15min":1.947},"last":0.973},{"interface":"front","average":{"1min":1.112,"5min":1.104,"15min":1.094},"min":{"1min":0.690,"5min":0.668,"15min":0.668},"max":{"1min":1.588,"5min":1.588,"15min":1.588},"last":1.165}]},{"osd":5,"last update":"Sun Dec 22 22:28:07 2019","interfaces":[{"interface":"back","average":{"1min":1.142,"5min":1.049,"15min":1.016},"min":{"1min":0.791,"5min":0.689,"15min":0.689},"max":{"1min":1.552,"5min":1.585,"15min":1.585},"last":1.209},{"interface":"front","average":{"1min":1.199,"5min":1.159,"15min":1.135},"min":{"1min":0.744,"5min":0.744,"15min":0.744},"max":{"1min":2.460,"5min":2.460,"15min":2.460},"last":1.499}]},{"osd":6,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.116,"5min":1.071,"15min":1.063},"min":{"1min":0.707,"5min":0.707,"15min":0.707},"max":{"1min":1.998,"5min":2.034,"15min":2.034},"last":0.932},{"interface":"front","average":{"1min":1.232,"5min":23.528,"15min":27.244},"min":{"1min":0.793,"5min":0.793,"15min":0.793},"max":{"1min":2.354,"5min":587.877,"15min":587.877},"last":1.384}]},{"osd":7,"last update":"Sun Dec 22 22:27:42 2019","interfaces":[{"interface":"back","average":{"1min":1.180,"5min":1.195,"15min":1.197},"min":{"1min":0.730,"5min":0.730,"15min":0.730},"max":{"1min":2.066,"5min":2.066,"15min":2.066},"last":1.030},{"interface":"front","average":{"1min":1.314,"5min":1.283,"15min":1.278},"min":{"1min":0.780,"5min":0.780,"15min":0.780},"max":{"1min":2.096,"5min":2.096,"15min":2.096},"last":1.608}]},{"osd":8,"last update":"Sun Dec 22 22:27:57 2019","interfaces":[{"interface":"back","average":{"1min":37.270,"5min":8.562,"15min":3.777},"min":{"1min":0.910,"5min":0.910,"15min":0.910},"max":{"1min":648.474,"5min":648.474,"15min":648.474},"last":1.245},{"interface":"front","average":{"1min":40.243,"5min":9.160,"15min":3.980},"min":{"1min":0.690,"5min":0.690,"15min":0.690},"max":{"1min":705.146,"5min":705.146,"15min":705.146},"last":1.131}]},{"osd":9,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":47.812,"5min":47.812,"15min":47.812},"min":{"1min":1.126,"5min":1.126,"15min":1.126},"max":{"1min":834.497,"5min":834.497,"15min":834.497},"last":1.540},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":1.017,"5min":1.017,"15min":1.017},"max":{"1min":1.749,"5min":1.749,"15min":1.749},"last":1.428}]},{"osd":10,"last update":"Sun Dec 22 22:27:42 2019","interfaces":[{"interface":"back","average":{"1min":59.261,"5min":59.261,"15min":59.261},"min":{"1min":1.117,"5min":1.117,"15min":1.117},"max":{"1min":984.644,"5min":984.644,"15min":984.644},"last":1.573},{"interface":"front","average":{"1min":1.477,"5min":1.477,"15min":1.477},"min":{"1min":0.897,"5min":0.897,"15min":0.897},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.329}]}]},{"osd":3,"up_from":0,"seq":54,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049236,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695916,"statfs":{"total":95995035648,"available":94920617984,"internally_reserved":1073741824,"allocated":585728,"data_stored":284195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:26:18 2019","interfaces":[{"interface":"back","average":{"1min":0.828,"5min":35.683,"15min":41.492},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.248,"5min":961.866,"15min":961.866},"last":1.531},{"interface":"front","average":{"1min":0.810,"5min":0.701,"15min":0.683},"min":{"1min":0.510,"5min":0.510,"15min":0.510},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.753}]},{"osd":1,"last update":"Sun Dec 22 22:27:21 2019","interfaces":[{"interface":"back","average":{"1min":0.865,"5min":0.740,"15min":0.674},"min":{"1min":0.512,"5min":0.498,"15min":0.498},"max":{"1min":1.178,"5min":2.091,"15min":2.091},"last":0.514},{"interface":"front","average":{"1min":28.452,"5min":6.305,"15min":2.590},"min":{"1min":0.407,"5min":0.407,"15min":0.407},"max":{"1min":495.556,"5min":495.556,"15min":495.556},"last":0.433}]},{"osd":2,"last update":"Sun Dec 22 22:27:21 2019","interfaces":[{"interface":"back","average":{"1min":5.722,"5min":23.239,"15min":32.091},"min":{"1min":0.473,"5min":0.473,"15min":0.473},"max":{"1min":87.773,"5min":750.955,"15min":750.955},"last":0.543},{"interface":"front","average":{"1min":49.085,"5min":10.532,"15min":4.098},"min":{"1min":0.541,"5min":0.541,"15min":0.541},"max":{"1min":866.045,"5min":866.045,"15min":866.045},"last":0.689}]},{"osd":4,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.186,"15min":1.186},"min":{"1min":0.730,"5min":0.730,"15min":0.730},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.762},{"interface":"front","average":{"1min":0.900,"5min":1.007,"15min":1.023},"min":{"1min":0.663,"5min":0.663,"15min":0.663},"max":{"1min":1.231,"5min":1.657,"15min":1.657},"last":0.849}]},{"osd":5,"last update":"Sun Dec 22 22:28:00 2019","interfaces":[{"interface":"back","average":{"1min":39.628,"5min":8.791,"15min":3.613},"min":{"1min":0.574,"5min":0.574,"15min":0.574},"max":{"1min":501.574,"5min":501.574,"15min":501.574},"last":0.916},{"interface":"front","average":{"1min":1.165,"5min":1.205,"15min":1.201},"min":{"1min":0.678,"5min":0.678,"15min":0.678},"max":{"1min":1.649,"5min":1.845,"15min":1.845},"last":0.879}]},{"osd":6,"last update":"Sun Dec 22 22:27:21 2019","interfaces":[{"interface":"back","average":{"1min":28.463,"5min":6.643,"15min":3.007},"min":{"1min":0.650,"5min":0.650,"15min":0.650},"max":{"1min":246.895,"5min":246.895,"15min":246.895},"last":0.816},{"interface":"front","average":{"1min":11.539,"5min":3.329,"15min":1.961},"min":{"1min":0.597,"5min":0.597,"15min":0.597},"max":{"1min":184.604,"5min":184.604,"15min":184.604},"last":0.946}]},{"osd":7,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":14.350,"15min":16.570},"min":{"1min":0.802,"5min":0.768,"15min":0.768},"max":{"1min":1.493,"5min":347.131,"15min":347.131},"last":0.994},{"interface":"front","average":{"1min":1.230,"5min":1.334,"15min":1.352},"min":{"1min":0.775,"5min":0.775,"15min":0.775},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":1.015}]},{"osd":8,"last update":"Sun Dec 22 22:28:00 2019","interfaces":[{"interface":"back","average":{"1min":1.372,"5min":1.411,"15min":1.417},"min":{"1min":0.885,"5min":0.885,"15min":0.885},"max":{"1min":2.279,"5min":2.279,"15min":2.279},"last":0.983},{"interface":"front","average":{"1min":1.366,"5min":1.420,"15min":1.429},"min":{"1min":0.865,"5min":0.865,"15min":0.865},"max":{"1min":1.838,"5min":2.137,"15min":2.137},"last":1.041}]},{"osd":9,"last update":"Sun Dec 22 22:27:21 2019","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":1.001,"5min":1.001,"15min":1.001},"max":{"1min":1.658,"5min":1.658,"15min":1.658},"last":1.130},{"interface":"front","average":{"1min":1.489,"5min":1.489,"15min":1.489},"min":{"1min":1.019,"5min":1.019,"15min":1.019},"max":{"1min":2.025,"5min":2.025,"15min":2.025},"last":1.148}]},{"osd":10,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":6.989,"5min":6.989,"15min":6.989},"min":{"1min":0.978,"5min":0.978,"15min":0.978},"max":{"1min":110.434,"5min":110.434,"15min":110.434},"last":1.180},{"interface":"front","average":{"1min":1.545,"5min":1.545,"15min":1.545},"min":{"1min":0.928,"5min":0.928,"15min":0.928},"max":{"1min":7.829,"5min":7.829,"15min":7.829},"last":1.088}]},{"osd":11,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.441,"5min":1.441,"15min":1.441},"min":{"1min":0.952,"5min":0.952,"15min":0.952},"max":{"1min":2.313,"5min":2.313,"15min":2.313},"last":1.111},{"interface":"front","average":{"1min":1.445,"5min":1.445,"15min":1.445},"min":{"1min":1.052,"5min":1.052,"15min":1.052},"max":{"1min":1.781,"5min":1.781,"15min":1.781},"last":1.244}]}]},{"osd":4,"up_from":0,"seq":49,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049236,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695916,"statfs":{"total":95995035648,"available":94920617984,"internally_reserved":1073741824,"allocated":585728,"data_stored":284195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":43.486,"5min":9.677,"15min":3.841},"min":{"1min":0.734,"5min":0.627,"15min":0.627},"max":{"1min":764.951,"5min":764.951,"15min":764.951},"last":0.995},{"interface":"front","average":{"1min":1.035,"5min":26.738,"15min":31.536},"min":{"1min":0.685,"5min":0.614,"15min":0.614},"max":{"1min":1.430,"5min":660.413,"15min":660.413},"last":1.035}]},{"osd":1,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":1.108,"5min":0.943,"15min":0.901},"min":{"1min":0.884,"5min":0.585,"15min":0.585},"max":{"1min":1.451,"5min":1.451,"15min":1.451},"last":1.181},{"interface":"front","average":{"1min":0.897,"5min":0.912,"15min":0.916},"min":{"1min":0.645,"5min":0.626,"15min":0.626},"max":{"1min":1.192,"5min":1.335,"15min":1.335},"last":1.114}]},{"osd":2,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":0.913,"5min":0.923,"15min":0.911},"min":{"1min":0.698,"5min":0.677,"15min":0.677},"max":{"1min":1.124,"5min":1.223,"15min":1.223},"last":949.510},{"interface":"front","average":{"1min":1.141,"5min":1.052,"15min":1.010},"min":{"1min":0.613,"5min":0.613,"15min":0.613},"max":{"1min":1.660,"5min":1.993,"15min":1.993},"last":1.148}]},{"osd":3,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":21.111,"5min":20.868,"15min":25.206},"min":{"1min":0.684,"5min":0.577,"15min":0.577},"max":{"1min":361.413,"5min":503.582,"15min":503.582},"last":0.808},{"interface":"front","average":{"1min":1.010,"5min":0.978,"15min":0.947},"min":{"1min":0.738,"5min":0.585,"15min":0.585},"max":{"1min":1.347,"5min":1.496,"15min":1.496},"last":1.381}]},{"osd":5,"last update":"Sun Dec 22 22:27:57 2019","interfaces":[{"interface":"back","average":{"1min":10.611,"5min":7.111,"15min":3.013},"min":{"1min":0.672,"5min":0.477,"15min":0.477},"max":{"1min":200.826,"5min":357.229,"15min":357.229},"last":0.857},{"interface":"front","average":{"1min":0.957,"5min":0.918,"15min":0.910},"min":{"1min":0.632,"5min":0.456,"15min":0.456},"max":{"1min":1.204,"5min":1.359,"15min":1.359},"last":0.912}]},{"osd":6,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":3.086,"15min":3.410},"min":{"1min":0.585,"5min":0.495,"15min":0.495},"max":{"1min":1.708,"5min":49.709,"15min":49.709},"last":1.342},{"interface":"front","average":{"1min":1.371,"5min":31.997,"15min":37.101},"min":{"1min":0.889,"5min":0.734,"15min":0.734},"max":{"1min":1.732,"5min":768.708,"15min":768.708},"last":1.427}]},{"osd":7,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.431,"15min":1.431},"min":{"1min":1.133,"5min":1.004,"15min":1.004},"max":{"1min":1.766,"5min":2.058,"15min":2.058},"last":1.518},{"interface":"front","average":{"1min":1.409,"5min":1.289,"15min":1.269},"min":{"1min":1.193,"5min":0.814,"15min":0.814},"max":{"1min":1.638,"5min":1.695,"15min":1.695},"last":1.479}]},{"osd":8,"last update":"Sun Dec 22 22:27:57 2019","interfaces":[{"interface":"back","average":{"1min":35.376,"5min":8.328,"15min":3.820},"min":{"1min":1.133,"5min":1.133,"15min":1.133},"max":{"1min":678.500,"5min":678.500,"15min":678.500},"last":1.784},{"interface":"front","average":{"1min":42.957,"5min":9.778,"15min":4.248},"min":{"1min":1.182,"5min":1.055,"15min":1.055},"max":{"1min":831.286,"5min":831.286,"15min":831.286},"last":1.580}]},{"osd":9,"last update":"Sun Dec 22 22:28:18 2019","interfaces":[{"interface":"back","average":{"1min":1.551,"5min":1.533,"15min":1.530},"min":{"1min":1.360,"5min":1.310,"15min":1.310},"max":{"1min":1.818,"5min":1.869,"15min":1.869},"last":1.696},{"interface":"front","average":{"1min":1.524,"5min":1.488,"15min":1.482},"min":{"1min":1.319,"5min":1.227,"15min":1.227},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":1.736}]},{"osd":10,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":18.270,"5min":18.270,"15min":18.270},"min":{"1min":1.313,"5min":1.313,"15min":1.313},"max":{"1min":302.146,"5min":302.146,"15min":302.146},"last":1.613},{"interface":"front","average":{"1min":1.541,"5min":1.541,"15min":1.541},"min":{"1min":1.340,"5min":1.340,"15min":1.340},"max":{"1min":1.767,"5min":1.767,"15min":1.767},"last":1.653}]}]},{"osd":5,"up_from":0,"seq":44,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049200,"kb_used_data":616,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695952,"statfs":{"total":95995035648,"available":94920654848,"internally_reserved":1073741824,"allocated":630784,"data_stored":288854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.189,"15min":1.258},"min":{"1min":0.611,"5min":0.557,"15min":0.557},"max":{"1min":2.727,"5min":7.646,"15min":7.646},"last":0.809},{"interface":"front","average":{"1min":1.145,"5min":22.091,"15min":31.454},"min":{"1min":0.743,"5min":0.548,"15min":0.548},"max":{"1min":2.694,"5min":803.966,"15min":803.966},"last":1.026}]},{"osd":1,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":12.150,"15min":17.025},"min":{"1min":0.679,"5min":0.553,"15min":0.553},"max":{"1min":8.832,"5min":456.064,"15min":456.064},"last":0.699},{"interface":"front","average":{"1min":1.067,"5min":1.152,"15min":1.219},"min":{"1min":0.632,"5min":0.619,"15min":0.619},"max":{"1min":1.543,"5min":7.728,"15min":7.728},"last":0.972}]},{"osd":2,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.406,"5min":8.846,"15min":12.250},"min":{"1min":0.658,"5min":0.651,"15min":0.651},"max":{"1min":8.927,"5min":319.127,"15min":319.127},"last":0.925},{"interface":"front","average":{"1min":1.141,"5min":1.268,"15min":1.332},"min":{"1min":0.777,"5min":0.722,"15min":0.722},"max":{"1min":2.257,"5min":7.975,"15min":7.975},"last":1.188}]},{"osd":3,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.515,"5min":8.301,"15min":3.706},"min":{"1min":0.720,"5min":0.659,"15min":0.659},"max":{"1min":8.877,"5min":660.469,"15min":660.469},"last":110.474},{"interface":"front","average":{"1min":36.220,"5min":8.352,"15min":3.754},"min":{"1min":0.784,"5min":0.784,"15min":0.784},"max":{"1min":455.681,"5min":455.681,"15min":455.681},"last":1.027}]},{"osd":4,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":0.989,"5min":33.395,"15min":47.786},"min":{"1min":0.562,"5min":0.537,"15min":0.537},"max":{"1min":1.550,"5min":886.107,"15min":886.107},"last":1.118},{"interface":"front","average":{"1min":1.078,"5min":1.266,"15min":1.349},"min":{"1min":0.667,"5min":0.526,"15min":0.526},"max":{"1min":1.415,"5min":11.065,"15min":11.065},"last":1.080}]},{"osd":6,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":14.251,"5min":3.795,"15min":2.053},"min":{"1min":0.933,"5min":0.583,"15min":0.583},"max":{"1min":260.820,"5min":260.820,"15min":260.820},"last":1.251},{"interface":"front","average":{"1min":1.293,"5min":1.245,"15min":1.237},"min":{"1min":0.679,"5min":0.612,"15min":0.612},"max":{"1min":2.292,"5min":2.292,"15min":2.292},"last":1.293}]},{"osd":7,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.257,"5min":1.237,"15min":1.233},"min":{"1min":0.891,"5min":0.841,"15min":0.841},"max":{"1min":1.672,"5min":1.672,"15min":1.672},"last":1.410},{"interface":"front","average":{"1min":1.465,"5min":1.333,"15min":1.311},"min":{"1min":1.148,"5min":0.802,"15min":0.802},"max":{"1min":2.166,"5min":2.175,"15min":2.175},"last":1.502}]},{"osd":8,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.484,"15min":1.476},"min":{"1min":0.982,"5min":0.982,"15min":0.982},"max":{"1min":2.315,"5min":2.315,"15min":2.315},"last":1.601},{"interface":"front","average":{"1min":1.618,"5min":5.511,"15min":6.160},"min":{"1min":1.196,"5min":1.196,"15min":1.196},"max":{"1min":2.334,"5min":91.314,"15min":91.314},"last":1.368}]},{"osd":9,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":1.204,"5min":1.204,"15min":1.204},"max":{"1min":1.880,"5min":1.880,"15min":1.880},"last":1.766},{"interface":"front","average":{"1min":1.543,"5min":1.543,"15min":1.543},"min":{"1min":1.177,"5min":1.177,"15min":1.177},"max":{"1min":2.430,"5min":2.430,"15min":2.430},"last":1.653}]},{"osd":10,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":1.135,"5min":1.135,"15min":1.135},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":1.737},{"interface":"front","average":{"1min":1.538,"5min":1.538,"15min":1.538},"min":{"1min":1.065,"5min":1.065,"15min":1.065},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.455}]}]}]}} 2019-12-22T22:28:23.298 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph pg dump --format=json 2019-12-22T22:28:25.570 INFO:teuthology.orchestra.run.smithi091.stdout: 2019-12-22T22:28:25.579 INFO:teuthology.orchestra.run.smithi091.stderr:dumped all 2019-12-22T22:28:26.081 INFO:teuthology.orchestra.run.smithi091.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2019-12-22T22:28:24.918674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":12,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1124941824,"kb_used":12590512,"kb_used_data":6864,"kb_used_omap":0,"kb_used_meta":12582912,"kb_avail":1112351312,"statfs":{"total":1151940427776,"available":1139047743488,"internally_reserved":12884901888,"allocated":7028736,"data_stored":3410340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12884901888},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[],"pool_stats":[],"osd_stats":[{"osd":11,"up_from":0,"seq":20,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049236,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695916,"statfs":{"total":95995035648,"available":94920617984,"internally_reserved":1073741824,"allocated":585728,"data_stored":284195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.099,"15min":1.099},"min":{"1min":0.618,"5min":0.618,"15min":0.618},"max":{"1min":2.561,"5min":2.561,"15min":2.561},"last":0.786},{"interface":"front","average":{"1min":0.962,"5min":0.962,"15min":0.962},"min":{"1min":0.641,"5min":0.641,"15min":0.641},"max":{"1min":1.942,"5min":1.942,"15min":1.942},"last":0.837}]},{"osd":1,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":0.856,"5min":0.856,"15min":0.856},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.037},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.707,"5min":0.707,"15min":0.707},"max":{"1min":3.452,"5min":3.452,"15min":3.452},"last":0.986}]},{"osd":2,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.037,"5min":1.037,"15min":1.037},"min":{"1min":0.812,"5min":0.812,"15min":0.812},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":1.170},{"interface":"front","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.656,"5min":0.656,"15min":0.656},"max":{"1min":1.312,"5min":1.312,"15min":1.312},"last":0.905}]},{"osd":3,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.687,"5min":0.687,"15min":0.687},"max":{"1min":1.666,"5min":1.666,"15min":1.666},"last":1.290},{"interface":"front","average":{"1min":1.335,"5min":1.335,"15min":1.335},"min":{"1min":0.692,"5min":0.692,"15min":0.692},"max":{"1min":2.688,"5min":2.688,"15min":2.688},"last":1.237}]},{"osd":4,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.533,"5min":1.533,"15min":1.533},"min":{"1min":0.955,"5min":0.955,"15min":0.955},"max":{"1min":3.629,"5min":3.629,"15min":3.629},"last":1.343},{"interface":"front","average":{"1min":1.270,"5min":1.270,"15min":1.270},"min":{"1min":0.761,"5min":0.761,"15min":0.761},"max":{"1min":1.894,"5min":1.894,"15min":1.894},"last":1.532}]},{"osd":5,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.341,"15min":1.341},"min":{"1min":0.820,"5min":0.820,"15min":0.820},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.381},{"interface":"front","average":{"1min":1.686,"5min":1.686,"15min":1.686},"min":{"1min":1.128,"5min":1.128,"15min":1.128},"max":{"1min":3.739,"5min":3.739,"15min":3.739},"last":1.436}]},{"osd":6,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":35.379,"5min":35.379,"15min":35.379},"min":{"1min":0.984,"5min":0.984,"15min":0.984},"max":{"1min":646.716,"5min":646.716,"15min":646.716},"last":1.652},{"interface":"front","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.905,"5min":0.905,"15min":0.905},"max":{"1min":2.199,"5min":2.199,"15min":2.199},"last":1.576}]},{"osd":7,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.558,"5min":1.558,"15min":1.558},"min":{"1min":1.116,"5min":1.116,"15min":1.116},"max":{"1min":2.283,"5min":2.283,"15min":2.283},"last":1.729},{"interface":"front","average":{"1min":1.566,"5min":1.566,"15min":1.566},"min":{"1min":1.204,"5min":1.204,"15min":1.204},"max":{"1min":2.399,"5min":2.399,"15min":2.399},"last":1.691}]},{"osd":8,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.822,"5min":1.822,"15min":1.822},"min":{"1min":1.237,"5min":1.237,"15min":1.237},"max":{"1min":3.936,"5min":3.936,"15min":3.936},"last":1.617},{"interface":"front","average":{"1min":34.440,"5min":34.440,"15min":34.440},"min":{"1min":1.332,"5min":1.332,"15min":1.332},"max":{"1min":624.904,"5min":624.904,"15min":624.904},"last":1.827}]},{"osd":10,"last update":"Sun Dec 22 22:27:56 2019","interfaces":[{"interface":"back","average":{"1min":1.871,"5min":1.871,"15min":1.871},"min":{"1min":1.205,"5min":1.205,"15min":1.205},"max":{"1min":4.041,"5min":4.041,"15min":4.041},"last":517.323},{"interface":"front","average":{"1min":1.772,"5min":1.772,"15min":1.772},"min":{"1min":1.354,"5min":1.354,"15min":1.354},"max":{"1min":3.823,"5min":3.823,"15min":3.823},"last":1.868}]}]},{"osd":10,"up_from":0,"seq":23,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049236,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695916,"statfs":{"total":95995035648,"available":94920617984,"internally_reserved":1073741824,"allocated":585728,"data_stored":284195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":1.404}]},{"osd":1,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.667,"5min":0.667,"15min":0.667},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.798},{"interface":"front","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.712,"5min":0.712,"15min":0.712},"max":{"1min":1.831,"5min":1.831,"15min":1.831},"last":0.861}]},{"osd":2,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":1.036,"15min":1.036},"min":{"1min":0.549,"5min":0.549,"15min":0.549},"max":{"1min":1.685,"5min":1.685,"15min":1.685},"last":1.079},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.604,"5min":0.604,"15min":0.604},"max":{"1min":2.191,"5min":2.191,"15min":2.191},"last":1.056}]},{"osd":3,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.010,"5min":1.010,"15min":1.010},"min":{"1min":0.635,"5min":0.635,"15min":0.635},"max":{"1min":1.438,"5min":1.438,"15min":1.438},"last":0.612},{"interface":"front","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.883,"5min":0.883,"15min":0.883},"max":{"1min":1.646,"5min":1.646,"15min":1.646},"last":0.992}]},{"osd":4,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.588,"5min":1.588,"15min":1.588},"min":{"1min":1.003,"5min":1.003,"15min":1.003},"max":{"1min":2.532,"5min":2.532,"15min":2.532},"last":1.570},{"interface":"front","average":{"1min":1.915,"5min":1.915,"15min":1.915},"min":{"1min":0.740,"5min":0.740,"15min":0.740},"max":{"1min":11.454,"5min":11.454,"15min":11.454},"last":1.373}]},{"osd":5,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.430,"5min":1.430,"15min":1.430},"min":{"1min":0.757,"5min":0.757,"15min":0.757},"max":{"1min":2.450,"5min":2.450,"15min":2.450},"last":0.760},{"interface":"front","average":{"1min":1.681,"5min":1.681,"15min":1.681},"min":{"1min":1.204,"5min":1.204,"15min":1.204},"max":{"1min":2.777,"5min":2.777,"15min":2.777},"last":1.169}]},{"osd":6,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":2.012,"5min":2.012,"15min":2.012},"min":{"1min":1.031,"5min":1.031,"15min":1.031},"max":{"1min":11.404,"5min":11.404,"15min":11.404},"last":1.135},{"interface":"front","average":{"1min":1.559,"5min":1.559,"15min":1.559},"min":{"1min":1.108,"5min":1.108,"15min":1.108},"max":{"1min":2.462,"5min":2.462,"15min":2.462},"last":1.404}]},{"osd":7,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":102.420,"5min":102.420,"15min":102.420},"min":{"1min":1.064,"5min":1.064,"15min":1.064},"max":{"1min":938.067,"5min":938.067,"15min":938.067},"last":1.204},{"interface":"front","average":{"1min":1.644,"5min":1.644,"15min":1.644},"min":{"1min":1.151,"5min":1.151,"15min":1.151},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":1.514}]},{"osd":8,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":2.393,"5min":2.393,"15min":2.393},"min":{"1min":1.068,"5min":1.068,"15min":1.068},"max":{"1min":11.238,"5min":11.238,"15min":11.238},"last":1.324},{"interface":"front","average":{"1min":2.286,"5min":2.286,"15min":2.286},"min":{"1min":0.928,"5min":0.928,"15min":0.928},"max":{"1min":11.482,"5min":11.482,"15min":11.482},"last":1.240}]},{"osd":9,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":2.448,"5min":2.448,"15min":2.448},"min":{"1min":1.242,"5min":1.242,"15min":1.242},"max":{"1min":11.328,"5min":11.328,"15min":11.328},"last":1.134},{"interface":"front","average":{"1min":2.308,"5min":2.308,"15min":2.308},"min":{"1min":1.182,"5min":1.182,"15min":1.182},"max":{"1min":11.191,"5min":11.191,"15min":11.191},"last":1.888}]},{"osd":11,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.723,"5min":1.723,"15min":1.723},"min":{"1min":1.306,"5min":1.306,"15min":1.306},"max":{"1min":2.336,"5min":2.336,"15min":2.336},"last":1.269},{"interface":"front","average":{"1min":1.915,"5min":1.915,"15min":1.915},"min":{"1min":1.594,"5min":1.594,"15min":1.594},"max":{"1min":2.620,"5min":2.620,"15min":2.620},"last":1.464}]}]},{"osd":9,"up_from":0,"seq":28,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049236,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695916,"statfs":{"total":95995035648,"available":94920617984,"internally_reserved":1073741824,"allocated":585728,"data_stored":284195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.084,"5min":1.084,"15min":1.084},"min":{"1min":0.636,"5min":0.636,"15min":0.636},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":1.188},{"interface":"front","average":{"1min":0.973,"5min":0.973,"15min":0.973},"min":{"1min":0.604,"5min":0.604,"15min":0.604},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.883}]},{"osd":1,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.037,"5min":1.037,"15min":1.037},"min":{"1min":0.656,"5min":0.656,"15min":0.656},"max":{"1min":1.664,"5min":1.664,"15min":1.664},"last":0.764},{"interface":"front","average":{"1min":0.891,"5min":0.891,"15min":0.891},"min":{"1min":0.647,"5min":0.647,"15min":0.647},"max":{"1min":1.437,"5min":1.437,"15min":1.437},"last":0.823}]},{"osd":2,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.197,"5min":1.197,"15min":1.197},"min":{"1min":0.738,"5min":0.738,"15min":0.738},"max":{"1min":2.162,"5min":2.162,"15min":2.162},"last":1.051},{"interface":"front","average":{"1min":27.186,"5min":27.186,"15min":27.186},"min":{"1min":0.668,"5min":0.668,"15min":0.668},"max":{"1min":524.094,"5min":524.094,"15min":524.094},"last":1.280}]},{"osd":3,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":35.175,"5min":35.175,"15min":35.175},"min":{"1min":0.538,"5min":0.538,"15min":0.538},"max":{"1min":683.851,"5min":683.851,"15min":683.851},"last":1.107},{"interface":"front","average":{"1min":1.256,"5min":1.256,"15min":1.256},"min":{"1min":0.599,"5min":0.599,"15min":0.599},"max":{"1min":2.067,"5min":2.067,"15min":2.067},"last":1.397}]},{"osd":4,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.219,"5min":1.219,"15min":1.219},"min":{"1min":0.840,"5min":0.840,"15min":0.840},"max":{"1min":1.574,"5min":1.574,"15min":1.574},"last":1.458},{"interface":"front","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.716,"5min":0.716,"15min":0.716},"max":{"1min":1.720,"5min":1.720,"15min":1.720},"last":1.494}]},{"osd":5,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.275,"5min":1.275,"15min":1.275},"min":{"1min":0.693,"5min":0.693,"15min":0.693},"max":{"1min":1.849,"5min":1.849,"15min":1.849},"last":1.341},{"interface":"front","average":{"1min":1.533,"5min":1.533,"15min":1.533},"min":{"1min":1.070,"5min":1.070,"15min":1.070},"max":{"1min":2.252,"5min":2.252,"15min":2.252},"last":1.693}]},{"osd":6,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":14.726,"5min":14.726,"15min":14.726},"min":{"1min":1.131,"5min":1.131,"15min":1.131},"max":{"1min":264.274,"5min":264.274,"15min":264.274},"last":1.629},{"interface":"front","average":{"1min":1.342,"5min":1.342,"15min":1.342},"min":{"1min":1.016,"5min":1.016,"15min":1.016},"max":{"1min":1.660,"5min":1.660,"15min":1.660},"last":1.569}]},{"osd":7,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.453,"5min":1.453,"15min":1.453},"min":{"1min":0.757,"5min":0.757,"15min":0.757},"max":{"1min":1.965,"5min":1.965,"15min":1.965},"last":1.795},{"interface":"front","average":{"1min":1.697,"5min":1.697,"15min":1.697},"min":{"1min":1.248,"5min":1.248,"15min":1.248},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":1.733}]},{"osd":8,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.443,"5min":1.443,"15min":1.443},"min":{"1min":0.700,"5min":0.700,"15min":0.700},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.670},{"interface":"front","average":{"1min":1.567,"5min":1.567,"15min":1.567},"min":{"1min":0.919,"5min":0.919,"15min":0.919},"max":{"1min":2.023,"5min":2.023,"15min":2.023},"last":372.462}]},{"osd":10,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.718,"5min":1.718,"15min":1.718},"min":{"1min":1.381,"5min":1.381,"15min":1.381},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":1.938},{"interface":"front","average":{"1min":1.802,"5min":1.802,"15min":1.802},"min":{"1min":1.407,"5min":1.407,"15min":1.407},"max":{"1min":2.188,"5min":2.188,"15min":2.188},"last":1.885}]}]},{"osd":8,"up_from":0,"seq":32,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049156,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695996,"statfs":{"total":95995035648,"available":94920699904,"internally_reserved":1073741824,"allocated":585728,"data_stored":284195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6,7,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.122,"5min":1.090,"15min":1.084},"min":{"1min":0.631,"5min":0.551,"15min":0.551},"max":{"1min":4.025,"5min":4.025,"15min":4.025},"last":0.937},{"interface":"front","average":{"1min":1.276,"5min":25.673,"15min":29.739},"min":{"1min":0.624,"5min":0.507,"15min":0.507},"max":{"1min":3.963,"5min":586.314,"15min":586.314},"last":1.157}]},{"osd":1,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":16.711,"5min":4.286,"15min":2.215},"min":{"1min":0.688,"5min":0.593,"15min":0.593},"max":{"1min":312.656,"5min":312.656,"15min":312.656},"last":1.335},{"interface":"front","average":{"1min":1.201,"5min":1.077,"15min":1.057},"min":{"1min":0.674,"5min":0.589,"15min":0.589},"max":{"1min":4.160,"5min":4.160,"15min":4.160},"last":0.988}]},{"osd":2,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.289,"5min":1.222,"15min":1.211},"min":{"1min":0.513,"5min":0.513,"15min":0.513},"max":{"1min":3.922,"5min":3.922,"15min":3.922},"last":0.521},{"interface":"front","average":{"1min":1.393,"5min":1.299,"15min":1.283},"min":{"1min":0.873,"5min":0.653,"15min":0.653},"max":{"1min":4.084,"5min":4.084,"15min":4.084},"last":1.266}]},{"osd":3,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.304,"15min":1.310},"min":{"1min":0.582,"5min":0.582,"15min":0.582},"max":{"1min":4.349,"5min":4.349,"15min":4.349},"last":0.658},{"interface":"front","average":{"1min":1.022,"5min":29.273,"15min":33.981},"min":{"1min":0.579,"5min":0.579,"15min":0.579},"max":{"1min":4.267,"5min":667.228,"15min":667.228},"last":1.221}]},{"osd":4,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.132,"5min":1.352,"15min":1.388},"min":{"1min":0.790,"5min":0.717,"15min":0.717},"max":{"1min":1.520,"5min":2.569,"15min":2.569},"last":1.030},{"interface":"front","average":{"1min":1.449,"5min":16.140,"15min":18.588},"min":{"1min":1.056,"5min":0.805,"15min":0.805},"max":{"1min":1.838,"5min":365.920,"15min":365.920},"last":1.439}]},{"osd":5,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.480,"15min":1.524},"min":{"1min":0.883,"5min":0.883,"15min":0.883},"max":{"1min":1.784,"5min":2.683,"15min":2.683},"last":0.856},{"interface":"front","average":{"1min":19.072,"5min":5.025,"15min":2.684},"min":{"1min":0.683,"5min":0.683,"15min":0.683},"max":{"1min":356.189,"5min":356.189,"15min":356.189},"last":1.490}]},{"osd":6,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.630,"15min":1.646},"min":{"1min":1.289,"5min":0.945,"15min":0.945},"max":{"1min":1.926,"5min":2.406,"15min":2.406},"last":849.814},{"interface":"front","average":{"1min":1.608,"5min":1.629,"15min":1.633},"min":{"1min":0.830,"5min":0.830,"15min":0.830},"max":{"1min":1.960,"5min":2.715,"15min":2.715},"last":1.529}]},{"osd":7,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":12.654,"15min":14.559},"min":{"1min":0.842,"5min":0.570,"15min":0.570},"max":{"1min":1.623,"5min":282.817,"15min":282.817},"last":1.111},{"interface":"front","average":{"1min":1.645,"5min":1.688,"15min":1.695},"min":{"1min":1.338,"5min":0.806,"15min":0.806},"max":{"1min":1.982,"5min":2.525,"15min":2.525},"last":1.300}]},{"osd":9,"last update":"Sun Dec 22 22:28:18 2019","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":42.310,"15min":49.110},"min":{"1min":0.701,"5min":0.525,"15min":0.525},"max":{"1min":2.146,"5min":686.274,"15min":686.274},"last":1.066},{"interface":"front","average":{"1min":1.563,"5min":1.652,"15min":1.667},"min":{"1min":1.089,"5min":0.464,"15min":0.464},"max":{"1min":2.043,"5min":2.587,"15min":2.587},"last":1.395}]},{"osd":10,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":19.663,"5min":19.663,"15min":19.663},"min":{"1min":0.732,"5min":0.732,"15min":0.732},"max":{"1min":358.731,"5min":358.731,"15min":358.731},"last":1.368},{"interface":"front","average":{"1min":1.829,"5min":1.829,"15min":1.829},"min":{"1min":0.843,"5min":0.843,"15min":0.843},"max":{"1min":2.791,"5min":2.791,"15min":2.791},"last":1.594}]}]},{"osd":7,"up_from":0,"seq":35,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049236,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695916,"statfs":{"total":95995035648,"available":94920617984,"internally_reserved":1073741824,"allocated":585728,"data_stored":284195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":0.973,"5min":0.854,"15min":0.834},"min":{"1min":0.558,"5min":0.558,"15min":0.558},"max":{"1min":1.443,"5min":1.443,"15min":1.443},"last":0.955},{"interface":"front","average":{"1min":14.982,"5min":3.818,"15min":1.958},"min":{"1min":0.731,"5min":0.684,"15min":0.684},"max":{"1min":291.908,"5min":291.908,"15min":291.908},"last":0.832}]},{"osd":1,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.197,"15min":1.201},"min":{"1min":0.819,"5min":0.758,"15min":0.758},"max":{"1min":1.616,"5min":2.308,"15min":2.308},"last":0.775},{"interface":"front","average":{"1min":1.079,"5min":1.059,"15min":1.056},"min":{"1min":0.680,"5min":0.680,"15min":0.680},"max":{"1min":1.396,"5min":1.937,"15min":1.937},"last":0.671}]},{"osd":2,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":0.991,"5min":1.018,"15min":1.022},"min":{"1min":0.685,"5min":0.685,"15min":0.685},"max":{"1min":1.529,"5min":1.886,"15min":1.886},"last":1.054},{"interface":"front","average":{"1min":1.121,"5min":1.145,"15min":1.149},"min":{"1min":0.817,"5min":0.625,"15min":0.625},"max":{"1min":1.524,"5min":2.376,"15min":2.376},"last":1.023}]},{"osd":3,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":42.500,"15min":49.405},"min":{"1min":0.648,"5min":0.648,"15min":0.648},"max":{"1min":1.494,"5min":985.876,"15min":985.876},"last":1.202},{"interface":"front","average":{"1min":1.217,"5min":1.272,"15min":1.281},"min":{"1min":0.784,"5min":0.694,"15min":0.694},"max":{"1min":1.559,"5min":2.153,"15min":2.153},"last":0.862}]},{"osd":4,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.143,"15min":1.151},"min":{"1min":0.699,"5min":0.600,"15min":0.600},"max":{"1min":1.654,"5min":1.800,"15min":1.800},"last":1.143},{"interface":"front","average":{"1min":1.159,"5min":1.191,"15min":1.196},"min":{"1min":0.704,"5min":0.704,"15min":0.704},"max":{"1min":1.643,"5min":1.833,"15min":1.833},"last":1.308}]},{"osd":5,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.154,"5min":1.273,"15min":1.293},"min":{"1min":0.633,"5min":0.633,"15min":0.633},"max":{"1min":1.629,"5min":1.663,"15min":1.663},"last":0.894},{"interface":"front","average":{"1min":1.398,"5min":1.387,"15min":1.385},"min":{"1min":0.855,"5min":0.728,"15min":0.728},"max":{"1min":1.747,"5min":2.435,"15min":2.435},"last":1.091}]},{"osd":6,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.322,"5min":1.273,"15min":1.265},"min":{"1min":0.611,"5min":0.611,"15min":0.611},"max":{"1min":1.710,"5min":2.106,"15min":2.106},"last":1.473},{"interface":"front","average":{"1min":1.369,"5min":31.793,"15min":36.864},"min":{"1min":0.976,"5min":0.976,"15min":0.976},"max":{"1min":1.755,"5min":684.801,"15min":684.801},"last":1.340}]},{"osd":8,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":29.540,"5min":7.085,"15min":3.343},"min":{"1min":0.901,"5min":0.807,"15min":0.807},"max":{"1min":502.217,"5min":502.217,"15min":502.217},"last":1.398},{"interface":"front","average":{"1min":1.560,"5min":1.628,"15min":1.639},"min":{"1min":1.038,"5min":1.038,"15min":1.038},"max":{"1min":1.788,"5min":2.619,"15min":2.619},"last":1.284}]},{"osd":9,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.606,"5min":1.606,"15min":1.606},"min":{"1min":1.228,"5min":1.228,"15min":1.228},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.434},{"interface":"front","average":{"1min":46.020,"5min":46.020,"15min":46.020},"min":{"1min":1.173,"5min":1.173,"15min":1.173},"max":{"1min":891.130,"5min":891.130,"15min":891.130},"last":1.568}]},{"osd":10,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":1.671,"5min":1.671,"15min":1.671},"min":{"1min":1.193,"5min":1.193,"15min":1.193},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.366},{"interface":"front","average":{"1min":53.065,"5min":53.065,"15min":53.065},"min":{"1min":1.327,"5min":1.327,"15min":1.327},"max":{"1min":748.773,"5min":748.773,"15min":748.773},"last":1.522}]}]},{"osd":6,"up_from":0,"seq":41,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049156,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695996,"statfs":{"total":95995035648,"available":94920699904,"internally_reserved":1073741824,"allocated":585728,"data_stored":284195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":1.053,"15min":1.083},"min":{"1min":0.601,"5min":0.601,"15min":0.601},"max":{"1min":1.682,"5min":1.682,"15min":1.682},"last":0.859},{"interface":"front","average":{"1min":0.974,"5min":1.074,"15min":1.090},"min":{"1min":0.689,"5min":0.689,"15min":0.689},"max":{"1min":1.528,"5min":1.677,"15min":1.677},"last":0.759}]},{"osd":1,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.893,"15min":0.895},"min":{"1min":0.626,"5min":0.591,"15min":0.591},"max":{"1min":1.364,"5min":1.793,"15min":1.793},"last":1.052},{"interface":"front","average":{"1min":0.920,"5min":19.041,"15min":22.061},"min":{"1min":0.611,"5min":0.588,"15min":0.588},"max":{"1min":1.628,"5min":363.766,"15min":363.766},"last":0.666}]},{"osd":2,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":5.483,"5min":1.961,"15min":1.374},"min":{"1min":0.674,"5min":0.674,"15min":0.674},"max":{"1min":89.000,"5min":89.000,"15min":89.000},"last":0.948},{"interface":"front","average":{"1min":1.033,"5min":46.178,"15min":53.702},"min":{"1min":0.721,"5min":0.721,"15min":0.721},"max":{"1min":1.550,"5min":904.141,"15min":904.141},"last":0.587}]},{"osd":3,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":16.675,"15min":19.261},"min":{"1min":0.902,"5min":0.834,"15min":0.834},"max":{"1min":1.511,"5min":312.765,"15min":312.765},"last":1.149},{"interface":"front","average":{"1min":1.160,"5min":1.184,"15min":1.188},"min":{"1min":0.719,"5min":0.719,"15min":0.719},"max":{"1min":1.468,"5min":1.771,"15min":1.771},"last":1.107}]},{"osd":4,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":47.470,"5min":10.303,"15min":4.109},"min":{"1min":0.727,"5min":0.535,"15min":0.535},"max":{"1min":882.404,"5min":882.404,"15min":882.404},"last":1.224},{"interface":"front","average":{"1min":1.110,"5min":1.036,"15min":1.024},"min":{"1min":0.653,"5min":0.595,"15min":0.595},"max":{"1min":1.572,"5min":1.572,"15min":1.572},"last":0.816}]},{"osd":5,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.197,"5min":1.164,"15min":1.158},"min":{"1min":0.843,"5min":0.680,"15min":0.680},"max":{"1min":1.453,"5min":1.839,"15min":1.839},"last":1.015},{"interface":"front","average":{"1min":1.296,"5min":1.257,"15min":1.251},"min":{"1min":0.960,"5min":0.896,"15min":0.896},"max":{"1min":1.706,"5min":1.804,"15min":1.804},"last":1.291}]},{"osd":7,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.311,"15min":1.307},"min":{"1min":1.073,"5min":1.030,"15min":1.030},"max":{"1min":1.590,"5min":1.675,"15min":1.675},"last":1.364},{"interface":"front","average":{"1min":1.392,"5min":1.350,"15min":1.343},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.656,"5min":1.721,"15min":1.721},"last":1.257}]},{"osd":8,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":49.165,"15min":57.102},"min":{"1min":1.168,"5min":1.168,"15min":1.168},"max":{"1min":1.972,"5min":844.789,"15min":844.789},"last":1.472},{"interface":"front","average":{"1min":1.408,"5min":1.431,"15min":1.435},"min":{"1min":1.200,"5min":1.183,"15min":1.183},"max":{"1min":1.840,"5min":1.840,"15min":1.840},"last":0.921}]},{"osd":9,"last update":"Sun Dec 22 22:27:18 2019","interfaces":[{"interface":"back","average":{"1min":1.531,"5min":1.531,"15min":1.531},"min":{"1min":1.113,"5min":1.113,"15min":1.113},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":1.392},{"interface":"front","average":{"1min":1.485,"5min":1.485,"15min":1.485},"min":{"1min":1.043,"5min":1.043,"15min":1.043},"max":{"1min":1.887,"5min":1.887,"15min":1.887},"last":1.184}]},{"osd":10,"last update":"Sun Dec 22 22:27:37 2019","interfaces":[{"interface":"back","average":{"1min":1.497,"5min":1.497,"15min":1.497},"min":{"1min":1.156,"5min":1.156,"15min":1.156},"max":{"1min":1.866,"5min":1.866,"15min":1.866},"last":1.441},{"interface":"front","average":{"1min":1.596,"5min":1.596,"15min":1.596},"min":{"1min":1.000,"5min":1.000,"15min":1.000},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.322}]}]},{"osd":1,"up_from":0,"seq":60,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049236,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695916,"statfs":{"total":95995035648,"available":94920617984,"internally_reserved":1073741824,"allocated":585728,"data_stored":284195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":0.805,"5min":0.645,"15min":0.595},"min":{"1min":0.525,"5min":0.338,"15min":0.338},"max":{"1min":1.279,"5min":1.279,"15min":1.279},"last":0.827},{"interface":"front","average":{"1min":0.901,"5min":0.829,"15min":0.735},"min":{"1min":0.507,"5min":0.381,"15min":0.381},"max":{"1min":1.370,"5min":1.511,"15min":1.511},"last":1.506}]},{"osd":2,"last update":"Sun Dec 22 22:28:05 2019","interfaces":[{"interface":"back","average":{"1min":1.011,"5min":0.885,"15min":0.797},"min":{"1min":0.500,"5min":0.391,"15min":0.391},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.631},{"interface":"front","average":{"1min":0.869,"5min":3.282,"15min":1.544},"min":{"1min":0.475,"5min":0.436,"15min":0.436},"max":{"1min":1.471,"5min":254.847,"15min":254.847},"last":0.979}]},{"osd":3,"last update":"Sun Dec 22 22:28:18 2019","interfaces":[{"interface":"back","average":{"1min":1.068,"5min":18.341,"15min":6.645},"min":{"1min":0.654,"5min":0.551,"15min":0.551},"max":{"1min":2.025,"5min":952.051,"15min":952.051},"last":1.382},{"interface":"front","average":{"1min":1.174,"5min":15.304,"15min":13.613},"min":{"1min":0.700,"5min":0.620,"15min":0.620},"max":{"1min":2.202,"5min":903.487,"15min":903.487},"last":1.756}]},{"osd":4,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":1.010,"5min":0.878,"15min":0.847},"min":{"1min":0.707,"5min":0.601,"15min":0.601},"max":{"1min":1.424,"5min":1.424,"15min":1.424},"last":1.131},{"interface":"front","average":{"1min":1.338,"5min":1.193,"15min":1.117},"min":{"1min":0.946,"5min":0.732,"15min":0.732},"max":{"1min":1.685,"5min":1.941,"15min":1.941},"last":1.694}]},{"osd":5,"last update":"Sun Dec 22 22:28:02 2019","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":0.977,"15min":0.957},"min":{"1min":0.851,"5min":0.676,"15min":0.676},"max":{"1min":1.521,"5min":1.521,"15min":1.521},"last":1.085},{"interface":"front","average":{"1min":12.769,"5min":3.568,"15min":2.000},"min":{"1min":0.809,"5min":0.809,"15min":0.809},"max":{"1min":263.068,"5min":263.068,"15min":263.068},"last":1.873}]},{"osd":6,"last update":"Sun Dec 22 22:27:23 2019","interfaces":[{"interface":"back","average":{"1min":1.410,"5min":1.411,"15min":1.411},"min":{"1min":0.972,"5min":0.972,"15min":0.972},"max":{"1min":2.000,"5min":2.315,"15min":2.315},"last":1.819},{"interface":"front","average":{"1min":1.019,"5min":1.013,"15min":1.012},"min":{"1min":0.736,"5min":0.736,"15min":0.736},"max":{"1min":1.527,"5min":1.673,"15min":1.673},"last":1.334}]},{"osd":7,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":1.400,"5min":1.473,"15min":1.485},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.861,"5min":2.238,"15min":2.238},"last":2.000},{"interface":"front","average":{"1min":6.449,"5min":2.532,"15min":1.879},"min":{"1min":0.963,"5min":0.963,"15min":0.963},"max":{"1min":101.788,"5min":101.788,"15min":101.788},"last":2.067}]},{"osd":8,"last update":"Sun Dec 22 22:28:02 2019","interfaces":[{"interface":"back","average":{"1min":1.189,"5min":1.189,"15min":1.189},"min":{"1min":0.826,"5min":0.826,"15min":0.826},"max":{"1min":1.594,"5min":1.594,"15min":1.594},"last":1.283},{"interface":"front","average":{"1min":1.196,"5min":1.228,"15min":1.233},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":1.566}]},{"osd":9,"last update":"Sun Dec 22 22:28:18 2019","interfaces":[{"interface":"back","average":{"1min":1.594,"5min":1.590,"15min":1.589},"min":{"1min":0.984,"5min":0.984,"15min":0.984},"max":{"1min":2.452,"5min":2.452,"15min":2.452},"last":2.184},{"interface":"front","average":{"1min":1.259,"5min":12.659,"15min":14.559},"min":{"1min":0.595,"5min":0.595,"15min":0.595},"max":{"1min":1.593,"5min":300.557,"15min":300.557},"last":1.459}]},{"osd":10,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":46.030,"5min":46.030,"15min":46.030},"min":{"1min":1.021,"5min":1.021,"15min":1.021},"max":{"1min":896.161,"5min":896.161,"15min":896.161},"last":1.228},{"interface":"front","average":{"1min":1.528,"5min":1.528,"15min":1.528},"min":{"1min":1.144,"5min":1.144,"15min":1.144},"max":{"1min":1.902,"5min":1.902,"15min":1.902},"last":2.128}]}]},{"osd":0,"up_from":0,"seq":65,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049156,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695996,"statfs":{"total":95995035648,"available":94920699904,"internally_reserved":1073741824,"allocated":585728,"data_stored":284195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":0.912,"5min":0.792,"15min":0.752},"min":{"1min":0.582,"5min":0.291,"15min":0.291},"max":{"1min":1.381,"5min":1.720,"15min":1.720},"last":0.812},{"interface":"front","average":{"1min":1.068,"5min":0.864,"15min":0.760},"min":{"1min":0.659,"5min":0.349,"15min":0.349},"max":{"1min":1.570,"5min":1.597,"15min":1.597},"last":0.940}]},{"osd":2,"last update":"Sun Dec 22 22:26:03 2019","interfaces":[{"interface":"back","average":{"1min":0.938,"5min":0.922,"15min":0.919},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.347,"5min":1.798,"15min":1.798},"last":1.188},{"interface":"front","average":{"1min":38.337,"5min":38.242,"15min":38.226},"min":{"1min":0.580,"5min":0.552,"15min":0.552},"max":{"1min":601.565,"5min":601.565,"15min":601.565},"last":0.645}]},{"osd":3,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.047,"5min":0.844,"15min":0.776},"min":{"1min":0.616,"5min":0.540,"15min":0.540},"max":{"1min":1.722,"5min":1.832,"15min":1.832},"last":1.074},{"interface":"front","average":{"1min":0.724,"5min":12.273,"15min":4.552},"min":{"1min":0.525,"5min":0.525,"15min":0.525},"max":{"1min":1.155,"5min":867.546,"15min":867.546},"last":0.701}]},{"osd":4,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":0.927,"5min":0.888,"15min":0.848},"min":{"1min":0.673,"5min":0.664,"15min":0.664},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":1.047},{"interface":"front","average":{"1min":1.313,"5min":1.149,"15min":1.077},"min":{"1min":0.945,"5min":0.850,"15min":0.850},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":1.150}]},{"osd":5,"last update":"Sun Dec 22 22:28:04 2019","interfaces":[{"interface":"back","average":{"1min":1.273,"5min":1.207,"15min":1.156},"min":{"1min":0.875,"5min":0.802,"15min":0.802},"max":{"1min":1.922,"5min":1.922,"15min":1.922},"last":1.206},{"interface":"front","average":{"1min":0.907,"5min":0.982,"15min":0.970},"min":{"1min":0.654,"5min":0.654,"15min":0.654},"max":{"1min":1.647,"5min":1.887,"15min":1.887},"last":1.001}]},{"osd":6,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.093,"5min":1.073,"15min":1.070},"min":{"1min":0.709,"5min":0.709,"15min":0.709},"max":{"1min":1.556,"5min":1.556,"15min":1.556},"last":0.880},{"interface":"front","average":{"1min":1.480,"5min":26.547,"15min":30.725},"min":{"1min":0.923,"5min":0.923,"15min":0.923},"max":{"1min":2.246,"5min":504.570,"15min":504.570},"last":1.516}]},{"osd":7,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.195,"15min":1.199},"min":{"1min":0.863,"5min":0.863,"15min":0.863},"max":{"1min":1.616,"5min":1.616,"15min":1.616},"last":1.113},{"interface":"front","average":{"1min":1.488,"5min":26.395,"15min":30.546},"min":{"1min":0.975,"5min":0.975,"15min":0.975},"max":{"1min":1.885,"5min":530.298,"15min":530.298},"last":1.592}]},{"osd":8,"last update":"Sun Dec 22 22:28:04 2019","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.303,"15min":1.319},"min":{"1min":0.815,"5min":0.805,"15min":0.805},"max":{"1min":1.771,"5min":1.771,"15min":1.771},"last":1.285},{"interface":"front","average":{"1min":46.831,"5min":10.652,"15min":4.622},"min":{"1min":0.892,"5min":0.892,"15min":0.892},"max":{"1min":817.964,"5min":817.964,"15min":817.964},"last":1.652}]},{"osd":9,"last update":"Sun Dec 22 22:28:17 2019","interfaces":[{"interface":"back","average":{"1min":1.554,"5min":1.630,"15min":1.642},"min":{"1min":1.004,"5min":1.004,"15min":1.004},"max":{"1min":2.055,"5min":2.180,"15min":2.180},"last":1.622},{"interface":"front","average":{"1min":1.152,"5min":1.241,"15min":1.256},"min":{"1min":0.729,"5min":0.729,"15min":0.729},"max":{"1min":1.523,"5min":1.925,"15min":1.925},"last":1.249}]},{"osd":10,"last update":"Sun Dec 22 22:27:40 2019","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.639,"15min":1.639},"min":{"1min":1.130,"5min":1.130,"15min":1.130},"max":{"1min":2.145,"5min":2.145,"15min":2.145},"last":1.686},{"interface":"front","average":{"1min":1.644,"5min":1.644,"15min":1.644},"min":{"1min":1.072,"5min":1.072,"15min":1.072},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":1.776}]},{"osd":11,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.160,"5min":1.160,"15min":1.160},"min":{"1min":0.772,"5min":0.772,"15min":0.772},"max":{"1min":1.712,"5min":1.712,"15min":1.712},"last":1.386},{"interface":"front","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":0.853,"5min":0.853,"15min":0.853},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.460}]}]},{"osd":2,"up_from":0,"seq":57,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049156,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695996,"statfs":{"total":95995035648,"available":94920699904,"internally_reserved":1073741824,"allocated":585728,"data_stored":284195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:28:07 2019","interfaces":[{"interface":"back","average":{"1min":0.733,"5min":0.657,"15min":0.567},"min":{"1min":0.418,"5min":0.398,"15min":0.398},"max":{"1min":1.422,"5min":1.461,"15min":1.461},"last":0.698},{"interface":"front","average":{"1min":0.832,"5min":0.699,"15min":0.623},"min":{"1min":0.423,"5min":0.423,"15min":0.423},"max":{"1min":1.684,"5min":1.684,"15min":1.684},"last":0.891}]},{"osd":1,"last update":"Sun Dec 22 22:28:07 2019","interfaces":[{"interface":"back","average":{"1min":0.909,"5min":0.819,"15min":0.745},"min":{"1min":0.586,"5min":0.501,"15min":0.501},"max":{"1min":1.324,"5min":1.555,"15min":1.555},"last":0.753},{"interface":"front","average":{"1min":1.038,"5min":0.810,"15min":0.744},"min":{"1min":0.656,"5min":0.478,"15min":0.478},"max":{"1min":1.406,"5min":1.406,"15min":1.406},"last":0.821}]},{"osd":3,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.029,"5min":0.868,"15min":0.818},"min":{"1min":0.711,"5min":0.563,"15min":0.563},"max":{"1min":1.261,"5min":1.261,"15min":1.261},"last":1.085},{"interface":"front","average":{"1min":0.952,"5min":0.840,"15min":0.818},"min":{"1min":0.611,"5min":0.525,"15min":0.525},"max":{"1min":1.930,"5min":1.930,"15min":1.930},"last":0.767}]},{"osd":4,"last update":"Sun Dec 22 22:27:42 2019","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.019,"15min":0.995},"min":{"1min":0.594,"5min":0.594,"15min":0.594},"max":{"1min":1.947,"5min":1.947,"15min":1.947},"last":0.973},{"interface":"front","average":{"1min":1.112,"5min":1.104,"15min":1.094},"min":{"1min":0.690,"5min":0.668,"15min":0.668},"max":{"1min":1.588,"5min":1.588,"15min":1.588},"last":1.165}]},{"osd":5,"last update":"Sun Dec 22 22:28:07 2019","interfaces":[{"interface":"back","average":{"1min":1.142,"5min":1.049,"15min":1.016},"min":{"1min":0.791,"5min":0.689,"15min":0.689},"max":{"1min":1.552,"5min":1.585,"15min":1.585},"last":1.209},{"interface":"front","average":{"1min":1.199,"5min":1.159,"15min":1.135},"min":{"1min":0.744,"5min":0.744,"15min":0.744},"max":{"1min":2.460,"5min":2.460,"15min":2.460},"last":1.499}]},{"osd":6,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.116,"5min":1.071,"15min":1.063},"min":{"1min":0.707,"5min":0.707,"15min":0.707},"max":{"1min":1.998,"5min":2.034,"15min":2.034},"last":0.932},{"interface":"front","average":{"1min":1.232,"5min":23.528,"15min":27.244},"min":{"1min":0.793,"5min":0.793,"15min":0.793},"max":{"1min":2.354,"5min":587.877,"15min":587.877},"last":1.384}]},{"osd":7,"last update":"Sun Dec 22 22:27:42 2019","interfaces":[{"interface":"back","average":{"1min":1.180,"5min":1.195,"15min":1.197},"min":{"1min":0.730,"5min":0.730,"15min":0.730},"max":{"1min":2.066,"5min":2.066,"15min":2.066},"last":1.030},{"interface":"front","average":{"1min":1.314,"5min":1.283,"15min":1.278},"min":{"1min":0.780,"5min":0.780,"15min":0.780},"max":{"1min":2.096,"5min":2.096,"15min":2.096},"last":1.608}]},{"osd":8,"last update":"Sun Dec 22 22:27:57 2019","interfaces":[{"interface":"back","average":{"1min":37.270,"5min":8.562,"15min":3.777},"min":{"1min":0.910,"5min":0.910,"15min":0.910},"max":{"1min":648.474,"5min":648.474,"15min":648.474},"last":1.245},{"interface":"front","average":{"1min":40.243,"5min":9.160,"15min":3.980},"min":{"1min":0.690,"5min":0.690,"15min":0.690},"max":{"1min":705.146,"5min":705.146,"15min":705.146},"last":1.131}]},{"osd":9,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":47.812,"5min":47.812,"15min":47.812},"min":{"1min":1.126,"5min":1.126,"15min":1.126},"max":{"1min":834.497,"5min":834.497,"15min":834.497},"last":1.540},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":1.017,"5min":1.017,"15min":1.017},"max":{"1min":1.749,"5min":1.749,"15min":1.749},"last":1.428}]},{"osd":10,"last update":"Sun Dec 22 22:27:42 2019","interfaces":[{"interface":"back","average":{"1min":59.261,"5min":59.261,"15min":59.261},"min":{"1min":1.117,"5min":1.117,"15min":1.117},"max":{"1min":984.644,"5min":984.644,"15min":984.644},"last":1.573},{"interface":"front","average":{"1min":1.477,"5min":1.477,"15min":1.477},"min":{"1min":0.897,"5min":0.897,"15min":0.897},"max":{"1min":2.048,"5min":2.048,"15min":2.048},"last":1.329}]}]},{"osd":3,"up_from":0,"seq":54,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049236,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695916,"statfs":{"total":95995035648,"available":94920617984,"internally_reserved":1073741824,"allocated":585728,"data_stored":284195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:26:18 2019","interfaces":[{"interface":"back","average":{"1min":0.828,"5min":35.683,"15min":41.492},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.248,"5min":961.866,"15min":961.866},"last":1.531},{"interface":"front","average":{"1min":0.810,"5min":0.701,"15min":0.683},"min":{"1min":0.510,"5min":0.510,"15min":0.510},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.753}]},{"osd":1,"last update":"Sun Dec 22 22:27:21 2019","interfaces":[{"interface":"back","average":{"1min":0.865,"5min":0.740,"15min":0.674},"min":{"1min":0.512,"5min":0.498,"15min":0.498},"max":{"1min":1.178,"5min":2.091,"15min":2.091},"last":0.514},{"interface":"front","average":{"1min":28.452,"5min":6.305,"15min":2.590},"min":{"1min":0.407,"5min":0.407,"15min":0.407},"max":{"1min":495.556,"5min":495.556,"15min":495.556},"last":0.433}]},{"osd":2,"last update":"Sun Dec 22 22:27:21 2019","interfaces":[{"interface":"back","average":{"1min":5.722,"5min":23.239,"15min":32.091},"min":{"1min":0.473,"5min":0.473,"15min":0.473},"max":{"1min":87.773,"5min":750.955,"15min":750.955},"last":0.543},{"interface":"front","average":{"1min":49.085,"5min":10.532,"15min":4.098},"min":{"1min":0.541,"5min":0.541,"15min":0.541},"max":{"1min":866.045,"5min":866.045,"15min":866.045},"last":0.689}]},{"osd":4,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.186,"15min":1.186},"min":{"1min":0.730,"5min":0.730,"15min":0.730},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.762},{"interface":"front","average":{"1min":0.900,"5min":1.007,"15min":1.023},"min":{"1min":0.663,"5min":0.663,"15min":0.663},"max":{"1min":1.231,"5min":1.657,"15min":1.657},"last":0.849}]},{"osd":5,"last update":"Sun Dec 22 22:28:00 2019","interfaces":[{"interface":"back","average":{"1min":39.628,"5min":8.791,"15min":3.613},"min":{"1min":0.574,"5min":0.574,"15min":0.574},"max":{"1min":501.574,"5min":501.574,"15min":501.574},"last":0.916},{"interface":"front","average":{"1min":1.165,"5min":1.205,"15min":1.201},"min":{"1min":0.678,"5min":0.678,"15min":0.678},"max":{"1min":1.649,"5min":1.845,"15min":1.845},"last":0.879}]},{"osd":6,"last update":"Sun Dec 22 22:27:21 2019","interfaces":[{"interface":"back","average":{"1min":28.463,"5min":6.643,"15min":3.007},"min":{"1min":0.650,"5min":0.650,"15min":0.650},"max":{"1min":246.895,"5min":246.895,"15min":246.895},"last":0.816},{"interface":"front","average":{"1min":11.539,"5min":3.329,"15min":1.961},"min":{"1min":0.597,"5min":0.597,"15min":0.597},"max":{"1min":184.604,"5min":184.604,"15min":184.604},"last":0.946}]},{"osd":7,"last update":"Sun Dec 22 22:27:44 2019","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":14.350,"15min":16.570},"min":{"1min":0.802,"5min":0.768,"15min":0.768},"max":{"1min":1.493,"5min":347.131,"15min":347.131},"last":0.994},{"interface":"front","average":{"1min":1.230,"5min":1.334,"15min":1.352},"min":{"1min":0.775,"5min":0.775,"15min":0.775},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":1.015}]},{"osd":8,"last update":"Sun Dec 22 22:28:00 2019","interfaces":[{"interface":"back","average":{"1min":1.372,"5min":1.411,"15min":1.417},"min":{"1min":0.885,"5min":0.885,"15min":0.885},"max":{"1min":2.279,"5min":2.279,"15min":2.279},"last":0.983},{"interface":"front","average":{"1min":1.366,"5min":1.420,"15min":1.429},"min":{"1min":0.865,"5min":0.865,"15min":0.865},"max":{"1min":1.838,"5min":2.137,"15min":2.137},"last":1.041}]},{"osd":9,"last update":"Sun Dec 22 22:27:21 2019","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.323,"15min":1.323},"min":{"1min":1.001,"5min":1.001,"15min":1.001},"max":{"1min":1.658,"5min":1.658,"15min":1.658},"last":1.130},{"interface":"front","average":{"1min":1.489,"5min":1.489,"15min":1.489},"min":{"1min":1.019,"5min":1.019,"15min":1.019},"max":{"1min":2.025,"5min":2.025,"15min":2.025},"last":1.148}]},{"osd":10,"last update":"Sun Dec 22 22:27:38 2019","interfaces":[{"interface":"back","average":{"1min":6.989,"5min":6.989,"15min":6.989},"min":{"1min":0.978,"5min":0.978,"15min":0.978},"max":{"1min":110.434,"5min":110.434,"15min":110.434},"last":1.180},{"interface":"front","average":{"1min":1.545,"5min":1.545,"15min":1.545},"min":{"1min":0.928,"5min":0.928,"15min":0.928},"max":{"1min":7.829,"5min":7.829,"15min":7.829},"last":1.088}]},{"osd":11,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.441,"5min":1.441,"15min":1.441},"min":{"1min":0.952,"5min":0.952,"15min":0.952},"max":{"1min":2.313,"5min":2.313,"15min":2.313},"last":1.111},{"interface":"front","average":{"1min":1.445,"5min":1.445,"15min":1.445},"min":{"1min":1.052,"5min":1.052,"15min":1.052},"max":{"1min":1.781,"5min":1.781,"15min":1.781},"last":1.244}]}]},{"osd":4,"up_from":0,"seq":49,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049236,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695916,"statfs":{"total":95995035648,"available":94920617984,"internally_reserved":1073741824,"allocated":585728,"data_stored":284195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":43.486,"5min":9.677,"15min":3.841},"min":{"1min":0.734,"5min":0.627,"15min":0.627},"max":{"1min":764.951,"5min":764.951,"15min":764.951},"last":0.995},{"interface":"front","average":{"1min":1.035,"5min":26.738,"15min":31.536},"min":{"1min":0.685,"5min":0.614,"15min":0.614},"max":{"1min":1.430,"5min":660.413,"15min":660.413},"last":1.035}]},{"osd":1,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":1.108,"5min":0.943,"15min":0.901},"min":{"1min":0.884,"5min":0.585,"15min":0.585},"max":{"1min":1.451,"5min":1.451,"15min":1.451},"last":1.181},{"interface":"front","average":{"1min":0.897,"5min":0.912,"15min":0.916},"min":{"1min":0.645,"5min":0.626,"15min":0.626},"max":{"1min":1.192,"5min":1.335,"15min":1.335},"last":1.114}]},{"osd":2,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":0.913,"5min":0.923,"15min":0.911},"min":{"1min":0.698,"5min":0.677,"15min":0.677},"max":{"1min":1.124,"5min":1.223,"15min":1.223},"last":949.510},{"interface":"front","average":{"1min":1.141,"5min":1.052,"15min":1.010},"min":{"1min":0.613,"5min":0.613,"15min":0.613},"max":{"1min":1.660,"5min":1.993,"15min":1.993},"last":1.148}]},{"osd":3,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":21.111,"5min":20.868,"15min":25.206},"min":{"1min":0.684,"5min":0.577,"15min":0.577},"max":{"1min":361.413,"5min":503.582,"15min":503.582},"last":0.808},{"interface":"front","average":{"1min":1.010,"5min":0.978,"15min":0.947},"min":{"1min":0.738,"5min":0.585,"15min":0.585},"max":{"1min":1.347,"5min":1.496,"15min":1.496},"last":1.381}]},{"osd":5,"last update":"Sun Dec 22 22:27:57 2019","interfaces":[{"interface":"back","average":{"1min":10.611,"5min":7.111,"15min":3.013},"min":{"1min":0.672,"5min":0.477,"15min":0.477},"max":{"1min":200.826,"5min":357.229,"15min":357.229},"last":0.857},{"interface":"front","average":{"1min":0.957,"5min":0.918,"15min":0.910},"min":{"1min":0.632,"5min":0.456,"15min":0.456},"max":{"1min":1.204,"5min":1.359,"15min":1.359},"last":0.912}]},{"osd":6,"last update":"Sun Dec 22 22:27:20 2019","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":3.086,"15min":3.410},"min":{"1min":0.585,"5min":0.495,"15min":0.495},"max":{"1min":1.708,"5min":49.709,"15min":49.709},"last":1.342},{"interface":"front","average":{"1min":1.371,"5min":31.997,"15min":37.101},"min":{"1min":0.889,"5min":0.734,"15min":0.734},"max":{"1min":1.732,"5min":768.708,"15min":768.708},"last":1.427}]},{"osd":7,"last update":"Sun Dec 22 22:27:41 2019","interfaces":[{"interface":"back","average":{"1min":1.427,"5min":1.431,"15min":1.431},"min":{"1min":1.133,"5min":1.004,"15min":1.004},"max":{"1min":1.766,"5min":2.058,"15min":2.058},"last":1.518},{"interface":"front","average":{"1min":1.409,"5min":1.289,"15min":1.269},"min":{"1min":1.193,"5min":0.814,"15min":0.814},"max":{"1min":1.638,"5min":1.695,"15min":1.695},"last":1.479}]},{"osd":8,"last update":"Sun Dec 22 22:27:57 2019","interfaces":[{"interface":"back","average":{"1min":35.376,"5min":8.328,"15min":3.820},"min":{"1min":1.133,"5min":1.133,"15min":1.133},"max":{"1min":678.500,"5min":678.500,"15min":678.500},"last":1.784},{"interface":"front","average":{"1min":42.957,"5min":9.778,"15min":4.248},"min":{"1min":1.182,"5min":1.055,"15min":1.055},"max":{"1min":831.286,"5min":831.286,"15min":831.286},"last":1.580}]},{"osd":9,"last update":"Sun Dec 22 22:28:18 2019","interfaces":[{"interface":"back","average":{"1min":1.551,"5min":1.533,"15min":1.530},"min":{"1min":1.360,"5min":1.310,"15min":1.310},"max":{"1min":1.818,"5min":1.869,"15min":1.869},"last":1.696},{"interface":"front","average":{"1min":1.524,"5min":1.488,"15min":1.482},"min":{"1min":1.319,"5min":1.227,"15min":1.227},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":1.736}]},{"osd":10,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":18.270,"5min":18.270,"15min":18.270},"min":{"1min":1.313,"5min":1.313,"15min":1.313},"max":{"1min":302.146,"5min":302.146,"15min":302.146},"last":1.613},{"interface":"front","average":{"1min":1.541,"5min":1.541,"15min":1.541},"min":{"1min":1.340,"5min":1.340,"15min":1.340},"max":{"1min":1.767,"5min":1.767,"15min":1.767},"last":1.653}]}]},{"osd":5,"up_from":0,"seq":45,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":1049236,"kb_used_data":572,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":92695916,"statfs":{"total":95995035648,"available":94920617984,"internally_reserved":1073741824,"allocated":585728,"data_stored":284195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.189,"15min":1.258},"min":{"1min":0.611,"5min":0.557,"15min":0.557},"max":{"1min":2.727,"5min":7.646,"15min":7.646},"last":0.676},{"interface":"front","average":{"1min":1.145,"5min":22.091,"15min":31.454},"min":{"1min":0.743,"5min":0.548,"15min":0.548},"max":{"1min":2.694,"5min":803.966,"15min":803.966},"last":291.398}]},{"osd":1,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":12.150,"15min":17.025},"min":{"1min":0.679,"5min":0.553,"15min":0.553},"max":{"1min":8.832,"5min":456.064,"15min":456.064},"last":0.970},{"interface":"front","average":{"1min":1.067,"5min":1.152,"15min":1.219},"min":{"1min":0.632,"5min":0.619,"15min":0.619},"max":{"1min":1.543,"5min":7.728,"15min":7.728},"last":1.073}]},{"osd":2,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.406,"5min":8.846,"15min":12.250},"min":{"1min":0.658,"5min":0.651,"15min":0.651},"max":{"1min":8.927,"5min":319.127,"15min":319.127},"last":0.780},{"interface":"front","average":{"1min":1.141,"5min":1.268,"15min":1.332},"min":{"1min":0.777,"5min":0.722,"15min":0.722},"max":{"1min":2.257,"5min":7.975,"15min":7.975},"last":0.875}]},{"osd":3,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":1.515,"5min":8.301,"15min":3.706},"min":{"1min":0.720,"5min":0.659,"15min":0.659},"max":{"1min":8.877,"5min":660.469,"15min":660.469},"last":0.924},{"interface":"front","average":{"1min":36.220,"5min":8.352,"15min":3.754},"min":{"1min":0.784,"5min":0.784,"15min":0.784},"max":{"1min":455.681,"5min":455.681,"15min":455.681},"last":1.149}]},{"osd":4,"last update":"Sun Dec 22 22:28:01 2019","interfaces":[{"interface":"back","average":{"1min":0.989,"5min":33.395,"15min":47.786},"min":{"1min":0.562,"5min":0.537,"15min":0.537},"max":{"1min":1.550,"5min":886.107,"15min":886.107},"last":1.367},{"interface":"front","average":{"1min":1.078,"5min":1.266,"15min":1.349},"min":{"1min":0.667,"5min":0.526,"15min":0.526},"max":{"1min":1.415,"5min":11.065,"15min":11.065},"last":1.213}]},{"osd":6,"last update":"Sun Dec 22 22:28:20 2019","interfaces":[{"interface":"back","average":{"1min":17.840,"5min":7.127,"15min":3.163},"min":{"1min":0.799,"5min":0.583,"15min":0.583},"max":{"1min":383.636,"5min":383.636,"15min":383.636},"last":1.403},{"interface":"front","average":{"1min":1.371,"5min":1.272,"15min":1.246},"min":{"1min":0.788,"5min":0.612,"15min":0.612},"max":{"1min":1.773,"5min":2.292,"15min":2.292},"last":1.322}]},{"osd":7,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.257,"5min":1.237,"15min":1.233},"min":{"1min":0.891,"5min":0.841,"15min":0.841},"max":{"1min":1.672,"5min":1.672,"15min":1.672},"last":1.255},{"interface":"front","average":{"1min":1.465,"5min":1.333,"15min":1.311},"min":{"1min":1.148,"5min":0.802,"15min":0.802},"max":{"1min":2.166,"5min":2.175,"15min":2.175},"last":1.536}]},{"osd":8,"last update":"Sun Dec 22 22:27:59 2019","interfaces":[{"interface":"back","average":{"1min":1.532,"5min":1.484,"15min":1.476},"min":{"1min":0.982,"5min":0.982,"15min":0.982},"max":{"1min":2.315,"5min":2.315,"15min":2.315},"last":1.486},{"interface":"front","average":{"1min":1.618,"5min":5.511,"15min":6.160},"min":{"1min":1.196,"5min":1.196,"15min":1.196},"max":{"1min":2.334,"5min":91.314,"15min":91.314},"last":1.438}]},{"osd":9,"last update":"Sun Dec 22 22:28:20 2019","interfaces":[{"interface":"back","average":{"1min":1.497,"5min":1.504,"15min":1.505},"min":{"1min":1.104,"5min":1.104,"15min":1.104},"max":{"1min":1.766,"5min":1.880,"15min":1.880},"last":1.287},{"interface":"front","average":{"1min":9.556,"5min":3.145,"15min":2.077},"min":{"1min":1.384,"5min":1.177,"15min":1.177},"max":{"1min":184.655,"5min":184.655,"15min":184.655},"last":1.574}]},{"osd":10,"last update":"Sun Dec 22 22:27:36 2019","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":1.135,"5min":1.135,"15min":1.135},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":1.639},{"interface":"front","average":{"1min":1.538,"5min":1.538,"15min":1.538},"min":{"1min":1.065,"5min":1.065,"15min":1.065},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.604}]}]}]}} 2019-12-22T22:28:26.088 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2019-12-22T22:28:26.088 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd dump 2019-12-22T22:28:28.192 INFO:teuthology.orchestra.run.smithi091.stdout:epoch 74 2019-12-22T22:28:28.192 INFO:teuthology.orchestra.run.smithi091.stdout:fsid 147630da-2509-11ea-8284-001a4aab830c 2019-12-22T22:28:28.193 INFO:teuthology.orchestra.run.smithi091.stdout:created 2019-12-22T22:20:00.154953+0000 2019-12-22T22:28:28.193 INFO:teuthology.orchestra.run.smithi091.stdout:modified 2019-12-22T22:28:18.004963+0000 2019-12-22T22:28:28.193 INFO:teuthology.orchestra.run.smithi091.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2019-12-22T22:28:28.193 INFO:teuthology.orchestra.run.smithi091.stdout:crush_version 26 2019-12-22T22:28:28.194 INFO:teuthology.orchestra.run.smithi091.stdout:full_ratio 0.95 2019-12-22T22:28:28.194 INFO:teuthology.orchestra.run.smithi091.stdout:backfillfull_ratio 0.9 2019-12-22T22:28:28.194 INFO:teuthology.orchestra.run.smithi091.stdout:nearfull_ratio 0.85 2019-12-22T22:28:28.194 INFO:teuthology.orchestra.run.smithi091.stdout:require_min_compat_client jewel 2019-12-22T22:28:28.194 INFO:teuthology.orchestra.run.smithi091.stdout:min_compat_client jewel 2019-12-22T22:28:28.195 INFO:teuthology.orchestra.run.smithi091.stdout:require_osd_release octopus 2019-12-22T22:28:28.197 INFO:teuthology.orchestra.run.smithi091.stdout:max_osd 12 2019-12-22T22:28:28.197 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0 up in weight 1 up_from 7 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.91:6803/1 [v2:172.21.15.91:6804/1,v1:172.21.15.91:6805/1] exists,up 4520fba2-2eb9-4a60-8945-d2d3a0e5128d 2019-12-22T22:28:28.197 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.91:6811/1 [v2:172.21.15.91:6812/1,v1:172.21.15.91:6813/1] exists,up ff7b9947-09d3-4363-9073-f074112a0eaa 2019-12-22T22:28:28.197 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.91:6819/1 [v2:172.21.15.91:6820/1,v1:172.21.15.91:6821/1] exists,up ccdb35fb-e575-4f63-8f4a-3dc721d55812 2019-12-22T22:28:28.197 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.91:6827/1 [v2:172.21.15.91:6828/1,v1:172.21.15.91:6829/1] exists,up 5eac4685-b5d5-41e5-84d7-11702f96d78e 2019-12-22T22:28:28.197 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4 up in weight 1 up_from 27 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.107:6801/1 [v2:172.21.15.107:6802/1,v1:172.21.15.107:6803/1] exists,up 0c8c0189-2dd0-4a79-9e1f-8533d73b4480 2019-12-22T22:28:28.198 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5 up in weight 1 up_from 32 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.107:6809/1 [v2:172.21.15.107:6810/1,v1:172.21.15.107:6811/1] exists,up 3a1924d5-2253-435f-a199-2286ef771c27 2019-12-22T22:28:28.199 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6 up in weight 1 up_from 37 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.107:6817/1 [v2:172.21.15.107:6818/1,v1:172.21.15.107:6819/1] exists,up 56e1405c-eeda-4e92-b8de-b330fe2b9a7d 2019-12-22T22:28:28.199 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7 up in weight 1 up_from 42 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.107:6825/1 [v2:172.21.15.107:6826/1,v1:172.21.15.107:6827/1] exists,up 9a55297d-6a61-444a-a3b1-66293cb3dfe9 2019-12-22T22:28:28.199 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8 up in weight 1 up_from 47 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.43:6801/1 [v2:172.21.15.43:6802/1,v1:172.21.15.43:6803/1] exists,up 71cd6cef-0dd6-4744-bd1e-d407cc9f4e11 2019-12-22T22:28:28.199 INFO:teuthology.orchestra.run.smithi091.stdout:osd.9 up in weight 1 up_from 52 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.43:6809/1 [v2:172.21.15.43:6810/1,v1:172.21.15.43:6811/1] exists,up 7c1f2ce9-60f1-49ce-8bdb-1608d4a5b47a 2019-12-22T22:28:28.199 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10 up in weight 1 up_from 57 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.43:6817/1 [v2:172.21.15.43:6818/1,v1:172.21.15.43:6819/1] exists,up 81aebf87-7a3a-4dc2-af79-2f21da7a7832 2019-12-22T22:28:28.200 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11 up in weight 1 up_from 62 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.43:6825/1 [v2:172.21.15.43:6826/1,v1:172.21.15.43:6827/1] exists,up d025b7a0-a2cb-4769-9908-5d1a93507c1f 2019-12-22T22:28:28.200 INFO:teuthology.orchestra.run.smithi091.stdout:blacklist 172.21.15.91:6801/1 expires 2019-12-23T22:20:16.666965+0000 2019-12-22T22:28:28.200 INFO:teuthology.orchestra.run.smithi091.stdout:blacklist 172.21.15.91:6800/1 expires 2019-12-23T22:20:16.666965+0000 2019-12-22T22:28:28.697 INFO:tasks.cephadm.ceph_manager.ceph:['osd.0 up in weight 1 up_from 7 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.91:6803/1 [v2:172.21.15.91:6804/1,v1:172.21.15.91:6805/1] exists,up 4520fba2-2eb9-4a60-8945-d2d3a0e5128d', 'osd.1 up in weight 1 up_from 12 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.91:6811/1 [v2:172.21.15.91:6812/1,v1:172.21.15.91:6813/1] exists,up ff7b9947-09d3-4363-9073-f074112a0eaa', 'osd.2 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.91:6819/1 [v2:172.21.15.91:6820/1,v1:172.21.15.91:6821/1] exists,up ccdb35fb-e575-4f63-8f4a-3dc721d55812', 'osd.3 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.91:6827/1 [v2:172.21.15.91:6828/1,v1:172.21.15.91:6829/1] exists,up 5eac4685-b5d5-41e5-84d7-11702f96d78e', 'osd.4 up in weight 1 up_from 27 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.107:6801/1 [v2:172.21.15.107:6802/1,v1:172.21.15.107:6803/1] exists,up 0c8c0189-2dd0-4a79-9e1f-8533d73b4480', 'osd.5 up in weight 1 up_from 32 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.107:6809/1 [v2:172.21.15.107:6810/1,v1:172.21.15.107:6811/1] exists,up 3a1924d5-2253-435f-a199-2286ef771c27', 'osd.6 up in weight 1 up_from 37 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.107:6817/1 [v2:172.21.15.107:6818/1,v1:172.21.15.107:6819/1] exists,up 56e1405c-eeda-4e92-b8de-b330fe2b9a7d', 'osd.7 up in weight 1 up_from 42 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.107:6825/1 [v2:172.21.15.107:6826/1,v1:172.21.15.107:6827/1] exists,up 9a55297d-6a61-444a-a3b1-66293cb3dfe9', 'osd.8 up in weight 1 up_from 47 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.43:6801/1 [v2:172.21.15.43:6802/1,v1:172.21.15.43:6803/1] exists,up 71cd6cef-0dd6-4744-bd1e-d407cc9f4e11', 'osd.9 up in weight 1 up_from 52 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.43:6809/1 [v2:172.21.15.43:6810/1,v1:172.21.15.43:6811/1] exists,up 7c1f2ce9-60f1-49ce-8bdb-1608d4a5b47a', 'osd.10 up in weight 1 up_from 57 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.43:6817/1 [v2:172.21.15.43:6818/1,v1:172.21.15.43:6819/1] exists,up 81aebf87-7a3a-4dc2-af79-2f21da7a7832', 'osd.11 up in weight 1 up_from 62 up_thru 0 down_at 0 last_clean_interval [0,0) v1:172.21.15.43:6825/1 [v2:172.21.15.43:6826/1,v1:172.21.15.43:6827/1] exists,up d025b7a0-a2cb-4769-9908-5d1a93507c1f'] 2019-12-22T22:28:28.698 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon mon.a config show 2019-12-22T22:28:30.676 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:28:30.676 INFO:teuthology.orchestra.run.smithi091.stdout: "name": "mon.a", 2019-12-22T22:28:30.677 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster": "ceph", 2019-12-22T22:28:30.677 INFO:teuthology.orchestra.run.smithi091.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2019-12-22T22:28:30.677 INFO:teuthology.orchestra.run.smithi091.stdout: "admin_socket_mode": "", 2019-12-22T22:28:30.677 INFO:teuthology.orchestra.run.smithi091.stdout: "allow_ansi": "Terminal", 2019-12-22T22:28:30.678 INFO:teuthology.orchestra.run.smithi091.stdout: "auth_client_required": "cephx, none", 2019-12-22T22:28:30.678 INFO:teuthology.orchestra.run.smithi091.stdout: "auth_cluster_required": "cephx", 2019-12-22T22:28:30.678 INFO:teuthology.orchestra.run.smithi091.stdout: "auth_debug": "true", 2019-12-22T22:28:30.678 INFO:teuthology.orchestra.run.smithi091.stdout: "auth_mon_ticket_ttl": "43200.000000", 2019-12-22T22:28:30.678 INFO:teuthology.orchestra.run.smithi091.stdout: "auth_service_required": "cephx", 2019-12-22T22:28:30.679 INFO:teuthology.orchestra.run.smithi091.stdout: "auth_service_ticket_ttl": "3600.000000", 2019-12-22T22:28:30.679 INFO:teuthology.orchestra.run.smithi091.stdout: "auth_supported": "", 2019-12-22T22:28:30.679 INFO:teuthology.orchestra.run.smithi091.stdout: "bdev_aio": "true", 2019-12-22T22:28:30.679 INFO:teuthology.orchestra.run.smithi091.stdout: "bdev_aio_max_queue_depth": "1024", 2019-12-22T22:28:30.679 INFO:teuthology.orchestra.run.smithi091.stdout: "bdev_aio_poll_ms": "250", 2019-12-22T22:28:30.680 INFO:teuthology.orchestra.run.smithi091.stdout: "bdev_aio_reap_max": "16", 2019-12-22T22:28:30.680 INFO:teuthology.orchestra.run.smithi091.stdout: "bdev_async_discard": "false", 2019-12-22T22:28:30.680 INFO:teuthology.orchestra.run.smithi091.stdout: "bdev_block_size": "4096", 2019-12-22T22:28:30.681 INFO:teuthology.orchestra.run.smithi091.stdout: "bdev_debug_aio": "false", 2019-12-22T22:28:30.681 INFO:teuthology.orchestra.run.smithi091.stdout: "bdev_debug_aio_log_age": "5.000000", 2019-12-22T22:28:30.681 INFO:teuthology.orchestra.run.smithi091.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2019-12-22T22:28:30.681 INFO:teuthology.orchestra.run.smithi091.stdout: "bdev_debug_inflight_ios": "false", 2019-12-22T22:28:30.681 INFO:teuthology.orchestra.run.smithi091.stdout: "bdev_enable_discard": "false", 2019-12-22T22:28:30.681 INFO:teuthology.orchestra.run.smithi091.stdout: "bdev_inject_crash": "0", 2019-12-22T22:28:30.682 INFO:teuthology.orchestra.run.smithi091.stdout: "bdev_inject_crash_flush_delay": "2", 2019-12-22T22:28:30.682 INFO:teuthology.orchestra.run.smithi091.stdout: "bdev_nvme_retry_count": "-1", 2019-12-22T22:28:30.682 INFO:teuthology.orchestra.run.smithi091.stdout: "bdev_nvme_unbind_from_kernel": "false", 2019-12-22T22:28:30.682 INFO:teuthology.orchestra.run.smithi091.stdout: "bluefs_alloc_size": "1048576", 2019-12-22T22:28:30.682 INFO:teuthology.orchestra.run.smithi091.stdout: "bluefs_allocator": "bitmap", 2019-12-22T22:28:30.683 INFO:teuthology.orchestra.run.smithi091.stdout: "bluefs_buffered_io": "true", 2019-12-22T22:28:30.683 INFO:teuthology.orchestra.run.smithi091.stdout: "bluefs_compact_log_sync": "false", 2019-12-22T22:28:30.683 INFO:teuthology.orchestra.run.smithi091.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2019-12-22T22:28:30.683 INFO:teuthology.orchestra.run.smithi091.stdout: "bluefs_log_compact_min_size": "16777216", 2019-12-22T22:28:30.684 INFO:teuthology.orchestra.run.smithi091.stdout: "bluefs_log_replay_check_allocations": "true", 2019-12-22T22:28:30.684 INFO:teuthology.orchestra.run.smithi091.stdout: "bluefs_max_log_runway": "4194304", 2019-12-22T22:28:30.684 INFO:teuthology.orchestra.run.smithi091.stdout: "bluefs_max_prefetch": "1048576", 2019-12-22T22:28:30.684 INFO:teuthology.orchestra.run.smithi091.stdout: "bluefs_min_flush_size": "524288", 2019-12-22T22:28:30.684 INFO:teuthology.orchestra.run.smithi091.stdout: "bluefs_min_log_runway": "1048576", 2019-12-22T22:28:30.685 INFO:teuthology.orchestra.run.smithi091.stdout: "bluefs_preextend_wal_files": "true", 2019-12-22T22:28:30.685 INFO:teuthology.orchestra.run.smithi091.stdout: "bluefs_shared_alloc_size": "65536", 2019-12-22T22:28:30.685 INFO:teuthology.orchestra.run.smithi091.stdout: "bluefs_sync_write": "false", 2019-12-22T22:28:30.685 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2019-12-22T22:28:30.685 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2019-12-22T22:28:30.685 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_allocator": "bitmap", 2019-12-22T22:28:30.686 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2019-12-22T22:28:30.686 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2019-12-22T22:28:30.686 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2019-12-22T22:28:30.686 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_bitmapallocator_span_size": "1024", 2019-12-22T22:28:30.686 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_blobid_prealloc": "10240", 2019-12-22T22:28:30.687 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_block_create": "true", 2019-12-22T22:28:30.687 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_block_db_create": "false", 2019-12-22T22:28:30.690 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_block_db_path": "", 2019-12-22T22:28:30.690 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_block_db_size": "0", 2019-12-22T22:28:30.690 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_block_path": "", 2019-12-22T22:28:30.690 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_block_preallocate_file": "false", 2019-12-22T22:28:30.690 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_block_size": "107374182400", 2019-12-22T22:28:30.690 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_block_wal_create": "false", 2019-12-22T22:28:30.690 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_block_wal_path": "", 2019-12-22T22:28:30.691 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_block_wal_size": "100663296", 2019-12-22T22:28:30.691 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_bluefs": "true", 2019-12-22T22:28:30.691 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2019-12-22T22:28:30.691 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_bluefs_balance_interval": "1.000000", 2019-12-22T22:28:30.691 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_bluefs_db_compatibility": "true", 2019-12-22T22:28:30.691 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_bluefs_env_mirror": "false", 2019-12-22T22:28:30.691 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_bluefs_gift_ratio": "0.020000", 2019-12-22T22:28:30.692 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_bluefs_max_free": "10737418240", 2019-12-22T22:28:30.692 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_bluefs_max_ratio": "0.900000", 2019-12-22T22:28:30.692 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_bluefs_min": "1073741824", 2019-12-22T22:28:30.692 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_bluefs_min_free": "1073741824", 2019-12-22T22:28:30.692 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_bluefs_min_ratio": "0.020000", 2019-12-22T22:28:30.693 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_bluefs_reclaim_ratio": "0.200000", 2019-12-22T22:28:30.693 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_cache_autotune": "true", 2019-12-22T22:28:30.693 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_cache_autotune_interval": "5.000000", 2019-12-22T22:28:30.693 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_cache_kv_ratio": "0.400000", 2019-12-22T22:28:30.693 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_cache_meta_ratio": "0.400000", 2019-12-22T22:28:30.693 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_cache_size": "0", 2019-12-22T22:28:30.693 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_cache_size_hdd": "1073741824", 2019-12-22T22:28:30.693 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_cache_size_ssd": "3221225472", 2019-12-22T22:28:30.694 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_cache_trim_interval": "0.050000", 2019-12-22T22:28:30.694 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_cache_trim_max_skip_pinned": "64", 2019-12-22T22:28:30.694 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_cache_type": "2q", 2019-12-22T22:28:30.694 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_clone_cow": "true", 2019-12-22T22:28:30.694 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_compression_algorithm": "snappy", 2019-12-22T22:28:30.694 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_compression_max_blob_size": "0", 2019-12-22T22:28:30.694 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_compression_max_blob_size_hdd": "524288", 2019-12-22T22:28:30.695 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2019-12-22T22:28:30.695 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_compression_min_blob_size": "0", 2019-12-22T22:28:30.695 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_compression_min_blob_size_hdd": "131072", 2019-12-22T22:28:30.695 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_compression_min_blob_size_ssd": "8192", 2019-12-22T22:28:30.695 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_compression_mode": "none", 2019-12-22T22:28:30.695 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_compression_required_ratio": "0.875000", 2019-12-22T22:28:30.695 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_csum_type": "crc32c", 2019-12-22T22:28:30.696 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_debug_enforce_settings": "default", 2019-12-22T22:28:30.696 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_debug_freelist": "false", 2019-12-22T22:28:30.696 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_debug_fsck_abort": "false", 2019-12-22T22:28:30.696 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_debug_inject_bug21040": "false", 2019-12-22T22:28:30.696 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2019-12-22T22:28:30.696 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_debug_inject_read_err": "false", 2019-12-22T22:28:30.696 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_debug_misc": "false", 2019-12-22T22:28:30.697 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_debug_no_reuse_blocks": "false", 2019-12-22T22:28:30.697 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_debug_omit_block_device_write": "false", 2019-12-22T22:28:30.697 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_debug_omit_kv_commit": "false", 2019-12-22T22:28:30.697 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2019-12-22T22:28:30.697 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_debug_prefill": "0.000000", 2019-12-22T22:28:30.697 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_debug_prefragment_max": "1048576", 2019-12-22T22:28:30.697 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_debug_random_read_err": "0.000000", 2019-12-22T22:28:30.697 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2019-12-22T22:28:30.698 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_debug_small_allocations": "0", 2019-12-22T22:28:30.698 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2019-12-22T22:28:30.698 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_default_buffered_read": "true", 2019-12-22T22:28:30.698 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_default_buffered_write": "false", 2019-12-22T22:28:30.698 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_deferred_batch_ops": "0", 2019-12-22T22:28:30.698 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2019-12-22T22:28:30.699 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2019-12-22T22:28:30.699 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2019-12-22T22:28:30.699 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_extent_map_shard_max_size": "1200", 2019-12-22T22:28:30.699 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_extent_map_shard_min_size": "150", 2019-12-22T22:28:30.700 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_extent_map_shard_target_size": "500", 2019-12-22T22:28:30.700 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2019-12-22T22:28:30.700 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_freelist_blocks_per_key": "128", 2019-12-22T22:28:30.700 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2019-12-22T22:28:30.703 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2019-12-22T22:28:30.703 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_fsck_on_mkfs": "true", 2019-12-22T22:28:30.703 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2019-12-22T22:28:30.703 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_fsck_on_mount": "false", 2019-12-22T22:28:30.703 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_fsck_on_mount_deep": "false", 2019-12-22T22:28:30.703 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_fsck_on_umount": "false", 2019-12-22T22:28:30.703 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_fsck_on_umount_deep": "false", 2019-12-22T22:28:30.704 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_fsck_quick_fix_on_mount": "true", 2019-12-22T22:28:30.704 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_fsck_quick_fix_threads": "2", 2019-12-22T22:28:30.704 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2019-12-22T22:28:30.704 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_gc_enable_blob_threshold": "0", 2019-12-22T22:28:30.705 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_gc_enable_total_threshold": "0", 2019-12-22T22:28:30.705 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_ignore_data_csum": "false", 2019-12-22T22:28:30.705 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_ioring": "false", 2019-12-22T22:28:30.705 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_kvbackend": "rocksdb", 2019-12-22T22:28:30.705 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_log_collection_list_age": "60.000000", 2019-12-22T22:28:30.705 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2019-12-22T22:28:30.706 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_log_op_age": "5.000000", 2019-12-22T22:28:30.706 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_max_alloc_size": "0", 2019-12-22T22:28:30.706 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_max_blob_size": "0", 2019-12-22T22:28:30.706 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_max_blob_size_hdd": "524288", 2019-12-22T22:28:30.706 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_max_blob_size_ssd": "65536", 2019-12-22T22:28:30.706 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_max_defer_interval": "3.000000", 2019-12-22T22:28:30.706 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_max_deferred_txc": "32", 2019-12-22T22:28:30.707 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_min_alloc_size": "0", 2019-12-22T22:28:30.707 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_min_alloc_size_hdd": "4096", 2019-12-22T22:28:30.707 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_min_alloc_size_ssd": "4096", 2019-12-22T22:28:30.707 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_nid_prealloc": "1024", 2019-12-22T22:28:30.707 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_prefer_deferred_size": "0", 2019-12-22T22:28:30.707 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_prefer_deferred_size_hdd": "32768", 2019-12-22T22:28:30.707 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2019-12-22T22:28:30.708 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_retry_disk_reads": "3", 2019-12-22T22:28:30.708 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_rocksdb_cf": "false", 2019-12-22T22:28:30.708 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_rocksdb_cfs": "M= P= L=", 2019-12-22T22:28:30.708 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2", 2019-12-22T22:28:30.708 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_spdk_coremask": "0x1", 2019-12-22T22:28:30.709 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_spdk_io_sleep": "5", 2019-12-22T22:28:30.709 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_spdk_max_io_completion": "0", 2019-12-22T22:28:30.712 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_spdk_mem": "512", 2019-12-22T22:28:30.712 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_sync_submit_transaction": "false", 2019-12-22T22:28:30.712 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_throttle_bytes": "67108864", 2019-12-22T22:28:30.712 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_throttle_cost_per_io": "0", 2019-12-22T22:28:30.712 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2019-12-22T22:28:30.713 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2019-12-22T22:28:30.713 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2019-12-22T22:28:30.713 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_throttle_trace_rate": "0.000000", 2019-12-22T22:28:30.713 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_tracing": "false", 2019-12-22T22:28:30.713 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_volume_selection_policy": "rocksdb_original", 2019-12-22T22:28:30.713 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_volume_selection_reserved": "0", 2019-12-22T22:28:30.713 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2019-12-22T22:28:30.714 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2019-12-22T22:28:30.714 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_warn_on_legacy_statfs": "true", 2019-12-22T22:28:30.714 INFO:teuthology.orchestra.run.smithi091.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2019-12-22T22:28:30.714 INFO:teuthology.orchestra.run.smithi091.stdout: "cephadm_path": "/usr/sbin/cephadm", 2019-12-22T22:28:30.714 INFO:teuthology.orchestra.run.smithi091.stdout: "cephx_cluster_require_signatures": "false", 2019-12-22T22:28:30.714 INFO:teuthology.orchestra.run.smithi091.stdout: "cephx_cluster_require_version": "1", 2019-12-22T22:28:30.714 INFO:teuthology.orchestra.run.smithi091.stdout: "cephx_require_signatures": "false", 2019-12-22T22:28:30.714 INFO:teuthology.orchestra.run.smithi091.stdout: "cephx_require_version": "1", 2019-12-22T22:28:30.715 INFO:teuthology.orchestra.run.smithi091.stdout: "cephx_service_require_signatures": "false", 2019-12-22T22:28:30.715 INFO:teuthology.orchestra.run.smithi091.stdout: "cephx_service_require_version": "1", 2019-12-22T22:28:30.715 INFO:teuthology.orchestra.run.smithi091.stdout: "cephx_sign_messages": "true", 2019-12-22T22:28:30.715 INFO:teuthology.orchestra.run.smithi091.stdout: "chdir": "", 2019-12-22T22:28:30.715 INFO:teuthology.orchestra.run.smithi091.stdout: "client_acl_type": "", 2019-12-22T22:28:30.715 INFO:teuthology.orchestra.run.smithi091.stdout: "client_cache_mid": "0.750000", 2019-12-22T22:28:30.716 INFO:teuthology.orchestra.run.smithi091.stdout: "client_cache_size": "16384", 2019-12-22T22:28:30.716 INFO:teuthology.orchestra.run.smithi091.stdout: "client_caps_release_delay": "5", 2019-12-22T22:28:30.716 INFO:teuthology.orchestra.run.smithi091.stdout: "client_check_pool_perm": "true", 2019-12-22T22:28:30.716 INFO:teuthology.orchestra.run.smithi091.stdout: "client_debug_force_sync_read": "false", 2019-12-22T22:28:30.716 INFO:teuthology.orchestra.run.smithi091.stdout: "client_debug_getattr_caps": "false", 2019-12-22T22:28:30.717 INFO:teuthology.orchestra.run.smithi091.stdout: "client_debug_inject_tick_delay": "0", 2019-12-22T22:28:30.717 INFO:teuthology.orchestra.run.smithi091.stdout: "client_die_on_failed_dentry_invalidate": "true", 2019-12-22T22:28:30.717 INFO:teuthology.orchestra.run.smithi091.stdout: "client_die_on_failed_remount": "false", 2019-12-22T22:28:30.717 INFO:teuthology.orchestra.run.smithi091.stdout: "client_dirsize_rbytes": "true", 2019-12-22T22:28:30.718 INFO:teuthology.orchestra.run.smithi091.stdout: "client_force_lazyio": "false", 2019-12-22T22:28:30.718 INFO:teuthology.orchestra.run.smithi091.stdout: "client_inject_fixed_oldest_tid": "false", 2019-12-22T22:28:30.718 INFO:teuthology.orchestra.run.smithi091.stdout: "client_inject_release_failure": "false", 2019-12-22T22:28:30.718 INFO:teuthology.orchestra.run.smithi091.stdout: "client_max_inline_size": "4096", 2019-12-22T22:28:30.719 INFO:teuthology.orchestra.run.smithi091.stdout: "client_mds_namespace": "", 2019-12-22T22:28:30.719 INFO:teuthology.orchestra.run.smithi091.stdout: "client_metadata": "", 2019-12-22T22:28:30.719 INFO:teuthology.orchestra.run.smithi091.stdout: "client_mount_gid": "-1", 2019-12-22T22:28:30.719 INFO:teuthology.orchestra.run.smithi091.stdout: "client_mount_timeout": "300.000000", 2019-12-22T22:28:30.720 INFO:teuthology.orchestra.run.smithi091.stdout: "client_mount_uid": "-1", 2019-12-22T22:28:30.720 INFO:teuthology.orchestra.run.smithi091.stdout: "client_mountpoint": "/", 2019-12-22T22:28:30.720 INFO:teuthology.orchestra.run.smithi091.stdout: "client_notify_timeout": "10", 2019-12-22T22:28:30.720 INFO:teuthology.orchestra.run.smithi091.stdout: "client_oc": "true", 2019-12-22T22:28:30.720 INFO:teuthology.orchestra.run.smithi091.stdout: "client_oc_max_dirty": "104857600", 2019-12-22T22:28:30.721 INFO:teuthology.orchestra.run.smithi091.stdout: "client_oc_max_dirty_age": "5.000000", 2019-12-22T22:28:30.721 INFO:teuthology.orchestra.run.smithi091.stdout: "client_oc_max_objects": "1000", 2019-12-22T22:28:30.721 INFO:teuthology.orchestra.run.smithi091.stdout: "client_oc_size": "209715200", 2019-12-22T22:28:30.721 INFO:teuthology.orchestra.run.smithi091.stdout: "client_oc_target_dirty": "8388608", 2019-12-22T22:28:30.722 INFO:teuthology.orchestra.run.smithi091.stdout: "client_permissions": "true", 2019-12-22T22:28:30.722 INFO:teuthology.orchestra.run.smithi091.stdout: "client_quota_df": "true", 2019-12-22T22:28:30.722 INFO:teuthology.orchestra.run.smithi091.stdout: "client_readahead_max_bytes": "0", 2019-12-22T22:28:30.722 INFO:teuthology.orchestra.run.smithi091.stdout: "client_readahead_max_periods": "4", 2019-12-22T22:28:30.723 INFO:teuthology.orchestra.run.smithi091.stdout: "client_readahead_min": "131072", 2019-12-22T22:28:30.723 INFO:teuthology.orchestra.run.smithi091.stdout: "client_reconnect_stale": "false", 2019-12-22T22:28:30.724 INFO:teuthology.orchestra.run.smithi091.stdout: "client_snapdir": ".snap", 2019-12-22T22:28:30.724 INFO:teuthology.orchestra.run.smithi091.stdout: "client_tick_interval": "1.000000", 2019-12-22T22:28:30.724 INFO:teuthology.orchestra.run.smithi091.stdout: "client_trace": "", 2019-12-22T22:28:30.724 INFO:teuthology.orchestra.run.smithi091.stdout: "client_try_dentry_invalidate": "false", 2019-12-22T22:28:30.725 INFO:teuthology.orchestra.run.smithi091.stdout: "client_use_faked_inos": "false", 2019-12-22T22:28:30.725 INFO:teuthology.orchestra.run.smithi091.stdout: "client_use_random_mds": "false", 2019-12-22T22:28:30.725 INFO:teuthology.orchestra.run.smithi091.stdout: "clog_to_graylog": "false", 2019-12-22T22:28:30.725 INFO:teuthology.orchestra.run.smithi091.stdout: "clog_to_graylog_host": "127.0.0.1", 2019-12-22T22:28:30.725 INFO:teuthology.orchestra.run.smithi091.stdout: "clog_to_graylog_port": "12201", 2019-12-22T22:28:30.725 INFO:teuthology.orchestra.run.smithi091.stdout: "clog_to_monitors": "default=true", 2019-12-22T22:28:30.726 INFO:teuthology.orchestra.run.smithi091.stdout: "clog_to_syslog": "false", 2019-12-22T22:28:30.726 INFO:teuthology.orchestra.run.smithi091.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2019-12-22T22:28:30.726 INFO:teuthology.orchestra.run.smithi091.stdout: "clog_to_syslog_level": "info", 2019-12-22T22:28:30.726 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_addr": "-", 2019-12-22T22:28:30.727 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_network": "", 2019-12-22T22:28:30.727 INFO:teuthology.orchestra.run.smithi091.stdout: "cluster_network_interface": "", 2019-12-22T22:28:30.727 INFO:teuthology.orchestra.run.smithi091.stdout: "colors": "Terminal", 2019-12-22T22:28:30.727 INFO:teuthology.orchestra.run.smithi091.stdout: "compressor_zlib_isal": "false", 2019-12-22T22:28:30.728 INFO:teuthology.orchestra.run.smithi091.stdout: "compressor_zlib_level": "5", 2019-12-22T22:28:30.728 INFO:teuthology.orchestra.run.smithi091.stdout: "container_image": "quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107", 2019-12-22T22:28:30.728 INFO:teuthology.orchestra.run.smithi091.stdout: "continuation_prompt": ">", 2019-12-22T22:28:30.729 INFO:teuthology.orchestra.run.smithi091.stdout: "crash_dir": "/var/lib/ceph/crash", 2019-12-22T22:28:30.729 INFO:teuthology.orchestra.run.smithi091.stdout: "crimson_osd_obc_lru_size": "10", 2019-12-22T22:28:30.729 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_location": "", 2019-12-22T22:28:30.729 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_location_hook": "", 2019-12-22T22:28:30.730 INFO:teuthology.orchestra.run.smithi091.stdout: "crush_location_hook_timeout": "10", 2019-12-22T22:28:30.730 INFO:teuthology.orchestra.run.smithi091.stdout: "daemonize": "false", 2019-12-22T22:28:30.731 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_allow_any_pool_priority": "false", 2019-12-22T22:28:30.731 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_asok": "1/5", 2019-12-22T22:28:30.731 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_asok_assert_abort": "false", 2019-12-22T22:28:30.731 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_asserts_on_shutdown": "true", 2019-12-22T22:28:30.732 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_auth": "1/5", 2019-12-22T22:28:30.732 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_bdev": "1/3", 2019-12-22T22:28:30.732 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_bluefs": "1/5", 2019-12-22T22:28:30.732 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_bluestore": "1/5", 2019-12-22T22:28:30.732 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_buffer": "0/1", 2019-12-22T22:28:30.733 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_civetweb": "1/10", 2019-12-22T22:28:30.733 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_client": "0/5", 2019-12-22T22:28:30.733 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_compressor": "1/5", 2019-12-22T22:28:30.733 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_context": "0/1", 2019-12-22T22:28:30.734 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_crush": "1/1", 2019-12-22T22:28:30.734 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_crypto": "1/5", 2019-12-22T22:28:30.734 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_deliberately_leak_memory": "false", 2019-12-22T22:28:30.734 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_disable_randomized_ping": "false", 2019-12-22T22:28:30.735 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_dpdk": "1/5", 2019-12-22T22:28:30.735 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_eventtrace": "1/5", 2019-12-22T22:28:30.735 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_filer": "0/1", 2019-12-22T22:28:30.735 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_filestore": "1/3", 2019-12-22T22:28:30.736 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_finisher": "1/1", 2019-12-22T22:28:30.736 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_fuse": "1/5", 2019-12-22T22:28:30.736 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_heartbeat_testing_span": "0", 2019-12-22T22:28:30.737 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_heartbeatmap": "1/5", 2019-12-22T22:28:30.737 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_immutable_obj_cache": "0/5", 2019-12-22T22:28:30.737 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_javaclient": "1/5", 2019-12-22T22:28:30.737 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_journal": "1/3", 2019-12-22T22:28:30.737 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_journaler": "0/5", 2019-12-22T22:28:30.738 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_kstore": "1/5", 2019-12-22T22:28:30.738 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_leveldb": "4/5", 2019-12-22T22:28:30.738 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_lockdep": "0/1", 2019-12-22T22:28:30.738 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_mds": "1/5", 2019-12-22T22:28:30.738 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_mds_balancer": "1/5", 2019-12-22T22:28:30.738 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_mds_locker": "1/5", 2019-12-22T22:28:30.738 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_mds_log": "1/5", 2019-12-22T22:28:30.738 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_mds_log_expire": "1/5", 2019-12-22T22:28:30.739 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_mds_migrator": "1/5", 2019-12-22T22:28:30.739 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_memdb": "4/5", 2019-12-22T22:28:30.739 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_mgr": "1/5", 2019-12-22T22:28:30.739 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_mgrc": "1/5", 2019-12-22T22:28:30.739 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_mon": "20/20", 2019-12-22T22:28:30.739 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_monc": "0/10", 2019-12-22T22:28:30.739 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_ms": "1/1", 2019-12-22T22:28:30.740 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_none": "0/5", 2019-12-22T22:28:30.740 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_objclass": "0/5", 2019-12-22T22:28:30.740 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_objectcacher": "0/5", 2019-12-22T22:28:30.740 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_objecter": "0/1", 2019-12-22T22:28:30.740 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_optracker": "0/5", 2019-12-22T22:28:30.740 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_osd": "1/5", 2019-12-22T22:28:30.740 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_paxos": "20/20", 2019-12-22T22:28:30.741 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_perfcounter": "1/5", 2019-12-22T22:28:30.741 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_prioritycache": "1/5", 2019-12-22T22:28:30.741 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_rados": "0/5", 2019-12-22T22:28:30.741 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_rbd": "0/5", 2019-12-22T22:28:30.741 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_rbd_mirror": "0/5", 2019-12-22T22:28:30.741 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_rbd_replay": "0/5", 2019-12-22T22:28:30.741 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_refs": "0/0", 2019-12-22T22:28:30.742 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_reserver": "1/1", 2019-12-22T22:28:30.742 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_rgw": "1/5", 2019-12-22T22:28:30.742 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_rgw_sync": "1/5", 2019-12-22T22:28:30.742 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_rocksdb": "4/5", 2019-12-22T22:28:30.742 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_shell": "false", 2019-12-22T22:28:30.742 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_striper": "0/1", 2019-12-22T22:28:30.742 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_test": "0/5", 2019-12-22T22:28:30.742 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_throttle": "1/1", 2019-12-22T22:28:30.743 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_timer": "0/1", 2019-12-22T22:28:30.743 INFO:teuthology.orchestra.run.smithi091.stdout: "debug_tp": "0/5", 2019-12-22T22:28:30.743 INFO:teuthology.orchestra.run.smithi091.stdout: "device_failure_prediction_mode": "none", 2019-12-22T22:28:30.743 INFO:teuthology.orchestra.run.smithi091.stdout: "echo": "false", 2019-12-22T22:28:30.743 INFO:teuthology.orchestra.run.smithi091.stdout: "editor": "vim", 2019-12-22T22:28:30.743 INFO:teuthology.orchestra.run.smithi091.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2019-12-22T22:28:30.743 INFO:teuthology.orchestra.run.smithi091.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2019-12-22T22:28:30.744 INFO:teuthology.orchestra.run.smithi091.stdout: "err_to_graylog": "false", 2019-12-22T22:28:30.744 INFO:teuthology.orchestra.run.smithi091.stdout: "err_to_stderr": "true", 2019-12-22T22:28:30.744 INFO:teuthology.orchestra.run.smithi091.stdout: "err_to_syslog": "false", 2019-12-22T22:28:30.744 INFO:teuthology.orchestra.run.smithi091.stdout: "event_tracing": "false", 2019-12-22T22:28:30.744 INFO:teuthology.orchestra.run.smithi091.stdout: "fake_statfs_for_testing": "0", 2019-12-22T22:28:30.744 INFO:teuthology.orchestra.run.smithi091.stdout: "fatal_signal_handlers": "true", 2019-12-22T22:28:30.745 INFO:teuthology.orchestra.run.smithi091.stdout: "feedback_to_output": "false", 2019-12-22T22:28:30.745 INFO:teuthology.orchestra.run.smithi091.stdout: "filer_max_purge_ops": "10", 2019-12-22T22:28:30.745 INFO:teuthology.orchestra.run.smithi091.stdout: "filer_max_truncate_ops": "128", 2019-12-22T22:28:30.745 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_apply_finisher_threads": "1", 2019-12-22T22:28:30.745 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_blackhole": "false", 2019-12-22T22:28:30.745 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_btrfs_clone_range": "true", 2019-12-22T22:28:30.745 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_btrfs_snap": "true", 2019-12-22T22:28:30.746 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_caller_concurrency": "10", 2019-12-22T22:28:30.746 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_collect_device_partition_information": "true", 2019-12-22T22:28:30.746 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_commit_timeout": "600.000000", 2019-12-22T22:28:30.746 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_debug_inject_read_err": "false", 2019-12-22T22:28:30.746 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_debug_omap_check": "false", 2019-12-22T22:28:30.746 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_debug_random_read_err": "0.000000", 2019-12-22T22:28:30.746 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_debug_verify_split": "false", 2019-12-22T22:28:30.747 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_dump_file": "", 2019-12-22T22:28:30.747 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2019-12-22T22:28:30.747 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_expected_throughput_ops": "200.000000", 2019-12-22T22:28:30.747 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_fadvise": "true", 2019-12-22T22:28:30.747 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_fail_eio": "true", 2019-12-22T22:28:30.747 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_fd_cache_shards": "16", 2019-12-22T22:28:30.747 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_fd_cache_size": "128", 2019-12-22T22:28:30.748 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_fiemap": "false", 2019-12-22T22:28:30.748 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_fiemap_threshold": "4096", 2019-12-22T22:28:30.748 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_fsync_flushes_journal_data": "false", 2019-12-22T22:28:30.748 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_index_retry_probability": "0.000000", 2019-12-22T22:28:30.748 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_inject_stall": "0", 2019-12-22T22:28:30.748 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_journal_parallel": "false", 2019-12-22T22:28:30.748 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_journal_trailing": "false", 2019-12-22T22:28:30.749 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_journal_writeahead": "false", 2019-12-22T22:28:30.749 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_kill_at": "0", 2019-12-22T22:28:30.749 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_max_alloc_hint_size": "1048576", 2019-12-22T22:28:30.749 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_max_inline_xattr_size": "0", 2019-12-22T22:28:30.749 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2019-12-22T22:28:30.749 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_max_inline_xattr_size_other": "512", 2019-12-22T22:28:30.749 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2019-12-22T22:28:30.750 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_max_inline_xattrs": "0", 2019-12-22T22:28:30.750 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2019-12-22T22:28:30.750 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_max_inline_xattrs_other": "2", 2019-12-22T22:28:30.750 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_max_inline_xattrs_xfs": "10", 2019-12-22T22:28:30.750 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_max_sync_interval": "5.000000", 2019-12-22T22:28:30.750 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_max_xattr_value_size": "0", 2019-12-22T22:28:30.750 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2019-12-22T22:28:30.751 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_max_xattr_value_size_other": "1024", 2019-12-22T22:28:30.751 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2019-12-22T22:28:30.751 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_merge_threshold": "-10", 2019-12-22T22:28:30.751 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_min_sync_interval": "0.010000", 2019-12-22T22:28:30.751 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_odsync_write": "false", 2019-12-22T22:28:30.751 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_omap_backend": "rocksdb", 2019-12-22T22:28:30.751 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_omap_backend_path": "", 2019-12-22T22:28:30.751 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_omap_header_cache_size": "1024", 2019-12-22T22:28:30.752 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_ondisk_finisher_threads": "1", 2019-12-22T22:28:30.752 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_op_thread_suicide_timeout": "180", 2019-12-22T22:28:30.752 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_op_thread_timeout": "60", 2019-12-22T22:28:30.753 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_op_threads": "2", 2019-12-22T22:28:30.753 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_punch_hole": "false", 2019-12-22T22:28:30.753 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2019-12-22T22:28:30.754 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2019-12-22T22:28:30.754 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2019-12-22T22:28:30.755 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_queue_high_threshhold": "0.900000", 2019-12-22T22:28:30.755 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_queue_low_threshhold": "0.300000", 2019-12-22T22:28:30.756 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_queue_max_bytes": "104857600", 2019-12-22T22:28:30.756 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2019-12-22T22:28:30.756 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2019-12-22T22:28:30.756 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2019-12-22T22:28:30.757 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_queue_max_ops": "50", 2019-12-22T22:28:30.757 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2019-12-22T22:28:30.757 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_seek_data_hole": "false", 2019-12-22T22:28:30.757 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_sloppy_crc": "false", 2019-12-22T22:28:30.758 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_sloppy_crc_block_size": "65536", 2019-12-22T22:28:30.758 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_splice": "false", 2019-12-22T22:28:30.758 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_split_multiple": "2", 2019-12-22T22:28:30.758 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_split_rand_factor": "20", 2019-12-22T22:28:30.759 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_update_to": "1000", 2019-12-22T22:28:30.759 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2019-12-22T22:28:30.759 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2019-12-22T22:28:30.760 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2019-12-22T22:28:30.760 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2019-12-22T22:28:30.761 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2019-12-22T22:28:30.761 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2019-12-22T22:28:30.761 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_wbthrottle_enable": "true", 2019-12-22T22:28:30.762 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2019-12-22T22:28:30.762 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2019-12-22T22:28:30.762 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2019-12-22T22:28:30.762 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2019-12-22T22:28:30.763 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2019-12-22T22:28:30.763 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2019-12-22T22:28:30.763 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_xfs_extsize": "false", 2019-12-22T22:28:30.763 INFO:teuthology.orchestra.run.smithi091.stdout: "filestore_zfs_snap": "false", 2019-12-22T22:28:30.764 INFO:teuthology.orchestra.run.smithi091.stdout: "fio_dir": "/tmp/fio", 2019-12-22T22:28:30.764 INFO:teuthology.orchestra.run.smithi091.stdout: "fsid": "147630da-2509-11ea-8284-001a4aab830c", 2019-12-22T22:28:30.764 INFO:teuthology.orchestra.run.smithi091.stdout: "fuse_allow_other": "true", 2019-12-22T22:28:30.765 INFO:teuthology.orchestra.run.smithi091.stdout: "fuse_atomic_o_trunc": "true", 2019-12-22T22:28:30.765 INFO:teuthology.orchestra.run.smithi091.stdout: "fuse_big_writes": "false", 2019-12-22T22:28:30.766 INFO:teuthology.orchestra.run.smithi091.stdout: "fuse_debug": "false", 2019-12-22T22:28:30.766 INFO:teuthology.orchestra.run.smithi091.stdout: "fuse_default_permissions": "false", 2019-12-22T22:28:30.766 INFO:teuthology.orchestra.run.smithi091.stdout: "fuse_disable_pagecache": "false", 2019-12-22T22:28:30.767 INFO:teuthology.orchestra.run.smithi091.stdout: "fuse_max_write": "0", 2019-12-22T22:28:30.767 INFO:teuthology.orchestra.run.smithi091.stdout: "fuse_multithreaded": "true", 2019-12-22T22:28:30.767 INFO:teuthology.orchestra.run.smithi091.stdout: "fuse_require_active_mds": "true", 2019-12-22T22:28:30.767 INFO:teuthology.orchestra.run.smithi091.stdout: "fuse_set_user_groups": "true", 2019-12-22T22:28:30.768 INFO:teuthology.orchestra.run.smithi091.stdout: "fuse_syncfs_on_mksnap": "true", 2019-12-22T22:28:30.768 INFO:teuthology.orchestra.run.smithi091.stdout: "fuse_use_invalidate_cb": "true", 2019-12-22T22:28:30.768 INFO:teuthology.orchestra.run.smithi091.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2019-12-22T22:28:30.768 INFO:teuthology.orchestra.run.smithi091.stdout: "gss_target_name": "ceph", 2019-12-22T22:28:30.769 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_file": "", 2019-12-22T22:28:30.769 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_inject_failure": "0", 2019-12-22T22:28:30.769 INFO:teuthology.orchestra.run.smithi091.stdout: "heartbeat_interval": "5", 2019-12-22T22:28:30.770 INFO:teuthology.orchestra.run.smithi091.stdout: "host": "", 2019-12-22T22:28:30.770 INFO:teuthology.orchestra.run.smithi091.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2019-12-22T22:28:30.770 INFO:teuthology.orchestra.run.smithi091.stdout: "immutable_object_cache_max_inflight_ops": "128", 2019-12-22T22:28:30.770 INFO:teuthology.orchestra.run.smithi091.stdout: "immutable_object_cache_max_size": "1073741824", 2019-12-22T22:28:30.771 INFO:teuthology.orchestra.run.smithi091.stdout: "immutable_object_cache_path": "/tmp", 2019-12-22T22:28:30.771 INFO:teuthology.orchestra.run.smithi091.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2019-12-22T22:28:30.771 INFO:teuthology.orchestra.run.smithi091.stdout: "immutable_object_cache_watermark": "0.100000", 2019-12-22T22:28:30.771 INFO:teuthology.orchestra.run.smithi091.stdout: "inject_early_sigterm": "false", 2019-12-22T22:28:30.772 INFO:teuthology.orchestra.run.smithi091.stdout: "journal_aio": "true", 2019-12-22T22:28:30.772 INFO:teuthology.orchestra.run.smithi091.stdout: "journal_align_min_size": "65536", 2019-12-22T22:28:30.772 INFO:teuthology.orchestra.run.smithi091.stdout: "journal_block_align": "true", 2019-12-22T22:28:30.772 INFO:teuthology.orchestra.run.smithi091.stdout: "journal_block_size": "4096", 2019-12-22T22:28:30.772 INFO:teuthology.orchestra.run.smithi091.stdout: "journal_dio": "true", 2019-12-22T22:28:30.773 INFO:teuthology.orchestra.run.smithi091.stdout: "journal_discard": "false", 2019-12-22T22:28:30.773 INFO:teuthology.orchestra.run.smithi091.stdout: "journal_force_aio": "false", 2019-12-22T22:28:30.773 INFO:teuthology.orchestra.run.smithi091.stdout: "journal_ignore_corruption": "false", 2019-12-22T22:28:30.773 INFO:teuthology.orchestra.run.smithi091.stdout: "journal_max_write_bytes": "10485760", 2019-12-22T22:28:30.774 INFO:teuthology.orchestra.run.smithi091.stdout: "journal_max_write_entries": "100", 2019-12-22T22:28:30.774 INFO:teuthology.orchestra.run.smithi091.stdout: "journal_replay_from": "0", 2019-12-22T22:28:30.774 INFO:teuthology.orchestra.run.smithi091.stdout: "journal_throttle_high_multiple": "0.000000", 2019-12-22T22:28:30.774 INFO:teuthology.orchestra.run.smithi091.stdout: "journal_throttle_high_threshhold": "0.900000", 2019-12-22T22:28:30.775 INFO:teuthology.orchestra.run.smithi091.stdout: "journal_throttle_low_threshhold": "0.600000", 2019-12-22T22:28:30.775 INFO:teuthology.orchestra.run.smithi091.stdout: "journal_throttle_max_multiple": "0.000000", 2019-12-22T22:28:30.775 INFO:teuthology.orchestra.run.smithi091.stdout: "journal_write_header_frequency": "0", 2019-12-22T22:28:30.775 INFO:teuthology.orchestra.run.smithi091.stdout: "journal_zero_on_create": "false", 2019-12-22T22:28:30.775 INFO:teuthology.orchestra.run.smithi091.stdout: "journaler_prefetch_periods": "10", 2019-12-22T22:28:30.775 INFO:teuthology.orchestra.run.smithi091.stdout: "journaler_prezero_periods": "5", 2019-12-22T22:28:30.776 INFO:teuthology.orchestra.run.smithi091.stdout: "journaler_write_head_interval": "15", 2019-12-22T22:28:30.776 INFO:teuthology.orchestra.run.smithi091.stdout: "key": "", 2019-12-22T22:28:30.777 INFO:teuthology.orchestra.run.smithi091.stdout: "keyfile": "", 2019-12-22T22:28:30.777 INFO:teuthology.orchestra.run.smithi091.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2019-12-22T22:28:30.777 INFO:teuthology.orchestra.run.smithi091.stdout: "kstore_backend": "rocksdb", 2019-12-22T22:28:30.777 INFO:teuthology.orchestra.run.smithi091.stdout: "kstore_default_stripe_size": "65536", 2019-12-22T22:28:30.777 INFO:teuthology.orchestra.run.smithi091.stdout: "kstore_fsck_on_mount": "false", 2019-12-22T22:28:30.777 INFO:teuthology.orchestra.run.smithi091.stdout: "kstore_fsck_on_mount_deep": "true", 2019-12-22T22:28:30.778 INFO:teuthology.orchestra.run.smithi091.stdout: "kstore_max_bytes": "67108864", 2019-12-22T22:28:30.778 INFO:teuthology.orchestra.run.smithi091.stdout: "kstore_max_ops": "512", 2019-12-22T22:28:30.778 INFO:teuthology.orchestra.run.smithi091.stdout: "kstore_nid_prealloc": "1024", 2019-12-22T22:28:30.778 INFO:teuthology.orchestra.run.smithi091.stdout: "kstore_onode_map_size": "1024", 2019-12-22T22:28:30.779 INFO:teuthology.orchestra.run.smithi091.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2019-12-22T22:28:30.779 INFO:teuthology.orchestra.run.smithi091.stdout: "kstore_sync_submit_transaction": "false", 2019-12-22T22:28:30.779 INFO:teuthology.orchestra.run.smithi091.stdout: "kstore_sync_transaction": "false", 2019-12-22T22:28:30.779 INFO:teuthology.orchestra.run.smithi091.stdout: "leveldb_block_size": "65536", 2019-12-22T22:28:30.779 INFO:teuthology.orchestra.run.smithi091.stdout: "leveldb_bloom_size": "0", 2019-12-22T22:28:30.780 INFO:teuthology.orchestra.run.smithi091.stdout: "leveldb_cache_size": "536870912", 2019-12-22T22:28:30.780 INFO:teuthology.orchestra.run.smithi091.stdout: "leveldb_compact_on_mount": "false", 2019-12-22T22:28:30.780 INFO:teuthology.orchestra.run.smithi091.stdout: "leveldb_compression": "false", 2019-12-22T22:28:30.780 INFO:teuthology.orchestra.run.smithi091.stdout: "leveldb_log": "", 2019-12-22T22:28:30.780 INFO:teuthology.orchestra.run.smithi091.stdout: "leveldb_log_to_ceph_log": "true", 2019-12-22T22:28:30.780 INFO:teuthology.orchestra.run.smithi091.stdout: "leveldb_max_open_files": "0", 2019-12-22T22:28:30.780 INFO:teuthology.orchestra.run.smithi091.stdout: "leveldb_paranoid": "false", 2019-12-22T22:28:30.781 INFO:teuthology.orchestra.run.smithi091.stdout: "leveldb_write_buffer_size": "33554432", 2019-12-22T22:28:30.781 INFO:teuthology.orchestra.run.smithi091.stdout: "lockdep": "false", 2019-12-22T22:28:30.781 INFO:teuthology.orchestra.run.smithi091.stdout: "lockdep_force_backtrace": "false", 2019-12-22T22:28:30.781 INFO:teuthology.orchestra.run.smithi091.stdout: "log_coarse_timestamps": "true", 2019-12-22T22:28:30.781 INFO:teuthology.orchestra.run.smithi091.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2019-12-22T22:28:30.781 INFO:teuthology.orchestra.run.smithi091.stdout: "log_flush_on_exit": "false", 2019-12-22T22:28:30.782 INFO:teuthology.orchestra.run.smithi091.stdout: "log_graylog_host": "127.0.0.1", 2019-12-22T22:28:30.782 INFO:teuthology.orchestra.run.smithi091.stdout: "log_graylog_port": "12201", 2019-12-22T22:28:30.782 INFO:teuthology.orchestra.run.smithi091.stdout: "log_max_new": "1000", 2019-12-22T22:28:30.782 INFO:teuthology.orchestra.run.smithi091.stdout: "log_max_recent": "10000", 2019-12-22T22:28:30.783 INFO:teuthology.orchestra.run.smithi091.stdout: "log_stderr_prefix": "", 2019-12-22T22:28:30.783 INFO:teuthology.orchestra.run.smithi091.stdout: "log_stop_at_utilization": "0.970000", 2019-12-22T22:28:30.783 INFO:teuthology.orchestra.run.smithi091.stdout: "log_to_file": "true", 2019-12-22T22:28:30.783 INFO:teuthology.orchestra.run.smithi091.stdout: "log_to_graylog": "false", 2019-12-22T22:28:30.784 INFO:teuthology.orchestra.run.smithi091.stdout: "log_to_stderr": "false", 2019-12-22T22:28:30.784 INFO:teuthology.orchestra.run.smithi091.stdout: "log_to_syslog": "false", 2019-12-22T22:28:30.784 INFO:teuthology.orchestra.run.smithi091.stdout: "max_completion_items": "50", 2019-12-22T22:28:30.784 INFO:teuthology.orchestra.run.smithi091.stdout: "max_rotating_auth_attempts": "10", 2019-12-22T22:28:30.784 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_action_on_write_error": "1", 2019-12-22T22:28:30.785 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_export_pin": "true", 2019-12-22T22:28:30.785 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_fragment_dirs": "true", 2019-12-22T22:28:30.785 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2019-12-22T22:28:30.785 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_fragment_interval": "5", 2019-12-22T22:28:30.786 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_fragment_size_max": "100000", 2019-12-22T22:28:30.786 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_idle_threshold": "0.000000", 2019-12-22T22:28:30.786 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_interval": "10", 2019-12-22T22:28:30.786 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_max": "-1", 2019-12-22T22:28:30.786 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_max_until": "-1", 2019-12-22T22:28:30.786 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_merge_size": "50", 2019-12-22T22:28:30.787 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_midchunk": "0.300000", 2019-12-22T22:28:30.787 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_min_rebalance": "0.100000", 2019-12-22T22:28:30.787 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_min_start": "0.200000", 2019-12-22T22:28:30.787 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_minchunk": "0.001000", 2019-12-22T22:28:30.787 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_mode": "0", 2019-12-22T22:28:30.788 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_need_max": "1.200000", 2019-12-22T22:28:30.788 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_need_min": "0.800000", 2019-12-22T22:28:30.788 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_replicate_threshold": "8000.000000", 2019-12-22T22:28:30.788 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_sample_interval": "3.000000", 2019-12-22T22:28:30.788 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_split_bits": "3", 2019-12-22T22:28:30.788 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_split_rd": "25000.000000", 2019-12-22T22:28:30.789 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_split_size": "10000", 2019-12-22T22:28:30.789 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_split_wr": "10000.000000", 2019-12-22T22:28:30.789 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_target_decay": "10.000000", 2019-12-22T22:28:30.789 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2019-12-22T22:28:30.789 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_beacon_grace": "15.000000", 2019-12-22T22:28:30.790 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_beacon_interval": "4.000000", 2019-12-22T22:28:30.790 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_cache_memory_limit": "4294967296", 2019-12-22T22:28:30.790 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_cache_mid": "0.700000", 2019-12-22T22:28:30.790 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_cache_release_free_interval": "10", 2019-12-22T22:28:30.790 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_cache_reservation": "0.050000", 2019-12-22T22:28:30.791 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_cache_trim_decay_rate": "1.000000", 2019-12-22T22:28:30.791 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_cache_trim_interval": "1", 2019-12-22T22:28:30.791 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_cache_trim_threshold": "65536", 2019-12-22T22:28:30.791 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2019-12-22T22:28:30.791 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_client_prealloc_inos": "1000", 2019-12-22T22:28:30.792 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2019-12-22T22:28:30.792 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_damage_table_max_entries": "10000", 2019-12-22T22:28:30.792 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2019-12-22T22:28:30.792 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_debug_auth_pins": "false", 2019-12-22T22:28:30.793 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_debug_frag": "false", 2019-12-22T22:28:30.793 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_debug_scatterstat": "false", 2019-12-22T22:28:30.794 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_debug_subtrees": "false", 2019-12-22T22:28:30.794 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_decay_halflife": "5.000000", 2019-12-22T22:28:30.794 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_default_dir_hash": "2", 2019-12-22T22:28:30.795 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_defer_session_stale": "true", 2019-12-22T22:28:30.795 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_dir_keys_per_op": "16384", 2019-12-22T22:28:30.795 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_dir_max_commit_size": "10", 2019-12-22T22:28:30.795 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_dirstat_min_interval": "1.000000", 2019-12-22T22:28:30.796 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_dump_cache_after_rejoin": "false", 2019-12-22T22:28:30.796 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_dump_cache_on_map": "false", 2019-12-22T22:28:30.796 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_dump_cache_threshold_file": "0", 2019-12-22T22:28:30.796 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2019-12-22T22:28:30.797 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_early_reply": "true", 2019-12-22T22:28:30.797 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_enable_op_tracker": "true", 2019-12-22T22:28:30.797 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_enforce_unique_name": "true", 2019-12-22T22:28:30.797 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_freeze_tree_timeout": "30.000000", 2019-12-22T22:28:30.797 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2019-12-22T22:28:30.798 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_health_cache_threshold": "1.500000", 2019-12-22T22:28:30.798 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_health_summarize_threshold": "10", 2019-12-22T22:28:30.798 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_heartbeat_grace": "15.000000", 2019-12-22T22:28:30.798 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_inject_migrator_session_race": "false", 2019-12-22T22:28:30.798 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2019-12-22T22:28:30.798 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_journal_format": "1", 2019-12-22T22:28:30.799 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_kill_create_at": "0", 2019-12-22T22:28:30.799 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_kill_export_at": "0", 2019-12-22T22:28:30.799 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_kill_import_at": "0", 2019-12-22T22:28:30.800 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_kill_journal_at": "0", 2019-12-22T22:28:30.800 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_kill_journal_expire_at": "0", 2019-12-22T22:28:30.800 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_kill_journal_replay_at": "0", 2019-12-22T22:28:30.800 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_kill_link_at": "0", 2019-12-22T22:28:30.801 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_kill_mdstable_at": "0", 2019-12-22T22:28:30.801 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_kill_openc_at": "0", 2019-12-22T22:28:30.802 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_kill_rename_at": "0", 2019-12-22T22:28:30.802 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_log_events_per_segment": "1024", 2019-12-22T22:28:30.802 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_log_max_events": "-1", 2019-12-22T22:28:30.802 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_log_max_segments": "128", 2019-12-22T22:28:30.802 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_log_pause": "false", 2019-12-22T22:28:30.803 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_log_segment_size": "0", 2019-12-22T22:28:30.803 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_log_skip_corrupt_events": "false", 2019-12-22T22:28:30.803 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_max_caps_per_client": "1048576", 2019-12-22T22:28:30.804 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_max_completed_flushes": "100000", 2019-12-22T22:28:30.804 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_max_completed_requests": "100000", 2019-12-22T22:28:30.804 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_max_export_size": "20971520", 2019-12-22T22:28:30.804 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_max_file_recover": "32", 2019-12-22T22:28:30.805 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_max_purge_files": "64", 2019-12-22T22:28:30.805 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_max_purge_ops": "8192", 2019-12-22T22:28:30.805 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2019-12-22T22:28:30.805 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_max_retries_on_remount_failure": "5", 2019-12-22T22:28:30.806 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_max_scrub_ops_in_progress": "5", 2019-12-22T22:28:30.806 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_max_snaps_per_dir": "100", 2019-12-22T22:28:30.806 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_max_xattr_pairs_size": "65536", 2019-12-22T22:28:30.806 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_min_caps_per_client": "100", 2019-12-22T22:28:30.806 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_mon_shutdown_timeout": "5.000000", 2019-12-22T22:28:30.807 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_numa_node": "-1", 2019-12-22T22:28:30.807 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_op_complaint_time": "30.000000", 2019-12-22T22:28:30.807 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_op_history_duration": "600", 2019-12-22T22:28:30.807 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_op_history_size": "20", 2019-12-22T22:28:30.807 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_op_log_threshold": "5", 2019-12-22T22:28:30.808 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2019-12-22T22:28:30.808 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_recall_global_max_decay_threshold": "65536", 2019-12-22T22:28:30.808 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_recall_max_caps": "5000", 2019-12-22T22:28:30.808 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_recall_max_decay_rate": "2.500000", 2019-12-22T22:28:30.809 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_recall_max_decay_threshold": "16384", 2019-12-22T22:28:30.809 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_recall_warning_decay_rate": "60.000000", 2019-12-22T22:28:30.809 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_recall_warning_threshold": "32768", 2019-12-22T22:28:30.809 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_reconnect_timeout": "45.000000", 2019-12-22T22:28:30.809 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_replay_interval": "1.000000", 2019-12-22T22:28:30.810 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_replay_unsafe_with_closed_session": "false", 2019-12-22T22:28:30.810 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_request_load_average_decay_rate": "60.000000", 2019-12-22T22:28:30.810 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_root_ino_gid": "0", 2019-12-22T22:28:30.810 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_root_ino_uid": "0", 2019-12-22T22:28:30.811 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_scatter_nudge_interval": "5.000000", 2019-12-22T22:28:30.811 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_session_blacklist_on_evict": "true", 2019-12-22T22:28:30.811 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_session_blacklist_on_timeout": "true", 2019-12-22T22:28:30.811 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2019-12-22T22:28:30.811 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_session_cache_liveness_magnitude": "10", 2019-12-22T22:28:30.812 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_sessionmap_keys_per_op": "1024", 2019-12-22T22:28:30.812 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_shutdown_check": "0", 2019-12-22T22:28:30.812 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_skip_ino": "0", 2019-12-22T22:28:30.812 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_snap_max_uid": "4294967294", 2019-12-22T22:28:30.812 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_snap_min_uid": "0", 2019-12-22T22:28:30.813 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_snap_rstat": "false", 2019-12-22T22:28:30.813 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_task_status_update_interval": "2.000000", 2019-12-22T22:28:30.813 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_thrash_exports": "0", 2019-12-22T22:28:30.813 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_thrash_fragments": "0", 2019-12-22T22:28:30.814 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_tick_interval": "5.000000", 2019-12-22T22:28:30.814 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_verify_backtrace": "1", 2019-12-22T22:28:30.814 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_verify_scatter": "false", 2019-12-22T22:28:30.814 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_wipe_ino_prealloc": "false", 2019-12-22T22:28:30.814 INFO:teuthology.orchestra.run.smithi091.stdout: "mds_wipe_sessions": "false", 2019-12-22T22:28:30.815 INFO:teuthology.orchestra.run.smithi091.stdout: "mempool_debug": "false", 2019-12-22T22:28:30.815 INFO:teuthology.orchestra.run.smithi091.stdout: "memstore_debug_omit_block_device_write": "false", 2019-12-22T22:28:30.815 INFO:teuthology.orchestra.run.smithi091.stdout: "memstore_device_bytes": "1073741824", 2019-12-22T22:28:30.815 INFO:teuthology.orchestra.run.smithi091.stdout: "memstore_page_set": "false", 2019-12-22T22:28:30.815 INFO:teuthology.orchestra.run.smithi091.stdout: "memstore_page_size": "65536", 2019-12-22T22:28:30.816 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr_client_bytes": "134217728", 2019-12-22T22:28:30.816 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr_client_messages": "512", 2019-12-22T22:28:30.816 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2019-12-22T22:28:30.817 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr_connect_retry_interval": "1.000000", 2019-12-22T22:28:30.817 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2019-12-22T22:28:30.819 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2019-12-22T22:28:30.819 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr_initial_modules": "restful iostat", 2019-12-22T22:28:30.819 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr_mds_bytes": "134217728", 2019-12-22T22:28:30.820 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr_mds_messages": "128", 2019-12-22T22:28:30.820 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2019-12-22T22:28:30.820 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr_mon_bytes": "134217728", 2019-12-22T22:28:30.820 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr_mon_messages": "128", 2019-12-22T22:28:30.821 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr_osd_bytes": "536870912", 2019-12-22T22:28:30.821 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr_osd_messages": "8192", 2019-12-22T22:28:30.821 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr_service_beacon_grace": "60.000000", 2019-12-22T22:28:30.821 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr_stats_period": "5", 2019-12-22T22:28:30.821 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr_stats_threshold": "5", 2019-12-22T22:28:30.822 INFO:teuthology.orchestra.run.smithi091.stdout: "mgr_tick_period": "2", 2019-12-22T22:28:30.822 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_accept_timeout_factor": "2.000000", 2019-12-22T22:28:30.823 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_allow_pool_delete": "true", 2019-12-22T22:28:30.824 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2019-12-22T22:28:30.824 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2019-12-22T22:28:30.824 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_client_bytes": "104857600", 2019-12-22T22:28:30.824 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_client_directed_command_retry": "5", 2019-12-22T22:28:30.825 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_client_hunt_interval": "3.000000", 2019-12-22T22:28:30.826 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2019-12-22T22:28:30.826 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2019-12-22T22:28:30.826 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2019-12-22T22:28:30.826 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_client_hunt_parallel": "3", 2019-12-22T22:28:30.826 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_client_max_log_entries_per_message": "1000", 2019-12-22T22:28:30.826 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_client_ping_interval": "10.000000", 2019-12-22T22:28:30.827 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_client_ping_timeout": "30.000000", 2019-12-22T22:28:30.827 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_clock_drift_allowed": "1.000000", 2019-12-22T22:28:30.827 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2019-12-22T22:28:30.827 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2019-12-22T22:28:30.827 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_cluster_log_file_level": "debug", 2019-12-22T22:28:30.827 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_cluster_log_to_file": "true", 2019-12-22T22:28:30.827 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_cluster_log_to_graylog": "false", 2019-12-22T22:28:30.827 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2019-12-22T22:28:30.828 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_cluster_log_to_graylog_port": "12201", 2019-12-22T22:28:30.828 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_cluster_log_to_stderr": "false", 2019-12-22T22:28:30.828 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_cluster_log_to_syslog": "default=false", 2019-12-22T22:28:30.828 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2019-12-22T22:28:30.828 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_cluster_log_to_syslog_level": "info", 2019-12-22T22:28:30.828 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_compact_on_bootstrap": "false", 2019-12-22T22:28:30.829 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_compact_on_start": "false", 2019-12-22T22:28:30.829 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_compact_on_trim": "true", 2019-12-22T22:28:30.829 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_config_key_max_entry_size": "65536", 2019-12-22T22:28:30.829 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_cpu_threads": "4", 2019-12-22T22:28:30.829 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_crush_min_required_version": "hammer", 2019-12-22T22:28:30.829 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_daemon_bytes": "419430400", 2019-12-22T22:28:30.829 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2019-12-22T22:28:30.829 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_data_avail_crit": "5", 2019-12-22T22:28:30.830 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_data_avail_warn": "5", 2019-12-22T22:28:30.830 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_data_size_warn": "16106127360", 2019-12-22T22:28:30.830 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_debug_block_osdmap_trim": "false", 2019-12-22T22:28:30.830 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_debug_deprecated_as_obsolete": "false", 2019-12-22T22:28:30.830 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_debug_dump_json": "false", 2019-12-22T22:28:30.830 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2019-12-22T22:28:30.830 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_debug_dump_transactions": "false", 2019-12-22T22:28:30.831 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_debug_extra_checks": "false", 2019-12-22T22:28:30.831 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_debug_no_initial_persistent_features": "false", 2019-12-22T22:28:30.831 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2019-12-22T22:28:30.831 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_debug_no_require_nautilus": "false", 2019-12-22T22:28:30.831 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_debug_no_require_octopus": "false", 2019-12-22T22:28:30.831 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2019-12-22T22:28:30.831 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_delta_reset_interval": "10.000000", 2019-12-22T22:28:30.832 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_dns_srv_name": "ceph-mon", 2019-12-22T22:28:30.832 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_election_timeout": "5.000000", 2019-12-22T22:28:30.832 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_enable_op_tracker": "true", 2019-12-22T22:28:30.832 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_fake_pool_delete": "false", 2019-12-22T22:28:30.832 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_force_quorum_join": "false", 2019-12-22T22:28:30.832 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_globalid_prealloc": "10000", 2019-12-22T22:28:30.832 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_health_log_update_period": "5", 2019-12-22T22:28:30.832 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_health_max_detail": "50", 2019-12-22T22:28:30.833 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_health_to_clog": "true", 2019-12-22T22:28:30.833 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_health_to_clog_interval": "3600", 2019-12-22T22:28:30.833 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2019-12-22T22:28:30.833 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_host": "172.21.15.91:6789/0", 2019-12-22T22:28:30.833 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_initial_members": "", 2019-12-22T22:28:30.833 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2019-12-22T22:28:30.833 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2019-12-22T22:28:30.834 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_inject_transaction_delay_max": "10.000000", 2019-12-22T22:28:30.834 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2019-12-22T22:28:30.834 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_keyvaluedb": "rocksdb", 2019-12-22T22:28:30.834 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_lease": "5.000000", 2019-12-22T22:28:30.834 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2019-12-22T22:28:30.834 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_lease_renew_interval_factor": "0.600000", 2019-12-22T22:28:30.834 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_log_max_summary": "50", 2019-12-22T22:28:30.835 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_max_log_entries_per_event": "4096", 2019-12-22T22:28:30.835 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_max_log_epochs": "500", 2019-12-22T22:28:30.835 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_max_mdsmap_epochs": "500", 2019-12-22T22:28:30.835 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_max_mgrmap_epochs": "500", 2019-12-22T22:28:30.835 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_max_osd": "10000", 2019-12-22T22:28:30.835 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_max_pg_per_osd": "10000", 2019-12-22T22:28:30.835 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_max_pool_pg_num": "65536", 2019-12-22T22:28:30.836 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_max_snap_prune_per_epoch": "100", 2019-12-22T22:28:30.836 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_mds_blacklist_interval": "86400.000000", 2019-12-22T22:28:30.836 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_mds_force_trim_to": "0", 2019-12-22T22:28:30.836 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_mds_skip_sanity": "false", 2019-12-22T22:28:30.836 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_memory_autotune": "true", 2019-12-22T22:28:30.836 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_memory_target": "2147483648", 2019-12-22T22:28:30.836 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_mgr_beacon_grace": "30", 2019-12-22T22:28:30.836 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_mgr_blacklist_interval": "86400.000000", 2019-12-22T22:28:30.837 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_mgr_digest_period": "5", 2019-12-22T22:28:30.837 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_mgr_inactive_grace": "60", 2019-12-22T22:28:30.837 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_mgr_mkfs_grace": "240", 2019-12-22T22:28:30.837 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2019-12-22T22:28:30.837 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_min_osdmap_epochs": "50", 2019-12-22T22:28:30.837 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_op_complaint_time": "30", 2019-12-22T22:28:30.837 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_op_history_duration": "600", 2019-12-22T22:28:30.838 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_op_history_size": "20", 2019-12-22T22:28:30.838 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_op_history_slow_op_size": "20", 2019-12-22T22:28:30.838 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_op_history_slow_op_threshold": "10", 2019-12-22T22:28:30.838 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_op_log_threshold": "5", 2019-12-22T22:28:30.838 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_adjust_down_out_interval": "true", 2019-12-22T22:28:30.838 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2019-12-22T22:28:30.838 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2019-12-22T22:28:30.839 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_auto_mark_in": "false", 2019-12-22T22:28:30.839 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_auto_mark_new_in": "true", 2019-12-22T22:28:30.839 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2019-12-22T22:28:30.839 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_blacklist_default_expire": "3600.000000", 2019-12-22T22:28:30.839 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_cache_size": "500", 2019-12-22T22:28:30.839 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_cache_size_min": "134217728", 2019-12-22T22:28:30.839 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_crush_smoke_test": "true", 2019-12-22T22:28:30.839 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_destroyed_out_interval": "600", 2019-12-22T22:28:30.840 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_down_out_interval": "600", 2019-12-22T22:28:30.840 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_down_out_subtree_limit": "rack", 2019-12-22T22:28:30.840 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2019-12-22T22:28:30.840 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_force_trim_to": "0", 2019-12-22T22:28:30.840 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_full_ratio": "0.950000", 2019-12-22T22:28:30.840 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_initial_require_min_compat_client": "jewel", 2019-12-22T22:28:30.840 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_laggy_halflife": "3600", 2019-12-22T22:28:30.841 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_laggy_max_interval": "300", 2019-12-22T22:28:30.841 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_laggy_weight": "0.300000", 2019-12-22T22:28:30.841 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2019-12-22T22:28:30.841 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_max_creating_pgs": "1024", 2019-12-22T22:28:30.841 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_max_initial_pgs": "1024", 2019-12-22T22:28:30.841 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_min_down_reporters": "2", 2019-12-22T22:28:30.841 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_min_in_ratio": "0.750000", 2019-12-22T22:28:30.842 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_min_up_ratio": "0.300000", 2019-12-22T22:28:30.842 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_nearfull_ratio": "0.850000", 2019-12-22T22:28:30.842 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_prime_pg_temp": "true", 2019-12-22T22:28:30.842 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2019-12-22T22:28:30.842 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2019-12-22T22:28:30.842 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_report_timeout": "900", 2019-12-22T22:28:30.842 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_reporter_subtree_level": "osd", 2019-12-22T22:28:30.842 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2019-12-22T22:28:30.843 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osd_warn_op_age": "32.000000", 2019-12-22T22:28:30.843 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osdmap_full_prune_enabled": "true", 2019-12-22T22:28:30.843 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osdmap_full_prune_interval": "2", 2019-12-22T22:28:30.843 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osdmap_full_prune_min": "15", 2019-12-22T22:28:30.843 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_osdmap_full_prune_txsize": "2", 2019-12-22T22:28:30.843 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2019-12-22T22:28:30.843 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_pg_stuck_threshold": "60", 2019-12-22T22:28:30.844 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2019-12-22T22:28:30.844 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_pg_warn_min_objects": "10000", 2019-12-22T22:28:30.844 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_pg_warn_min_per_osd": "0", 2019-12-22T22:28:30.844 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_pg_warn_min_pool_objects": "1000", 2019-12-22T22:28:30.844 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_pool_quota_crit_threshold": "0", 2019-12-22T22:28:30.844 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_pool_quota_warn_threshold": "0", 2019-12-22T22:28:30.844 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_probe_timeout": "2.000000", 2019-12-22T22:28:30.845 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_reweight_max_change": "0.050000", 2019-12-22T22:28:30.845 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_reweight_max_osds": "4", 2019-12-22T22:28:30.845 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_reweight_min_bytes_per_osd": "10", 2019-12-22T22:28:30.845 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_reweight_min_pgs_per_osd": "4", 2019-12-22T22:28:30.845 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2019-12-22T22:28:30.845 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2019-12-22T22:28:30.845 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2019-12-22T22:28:30.846 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_scrub_interval": "86400", 2019-12-22T22:28:30.846 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_scrub_max_keys": "100", 2019-12-22T22:28:30.846 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_scrub_timeout": "300", 2019-12-22T22:28:30.846 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_session_timeout": "300", 2019-12-22T22:28:30.846 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_smart_report_timeout": "5", 2019-12-22T22:28:30.846 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_stat_smooth_intervals": "6", 2019-12-22T22:28:30.846 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_subscribe_interval": "86400.000000", 2019-12-22T22:28:30.846 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_sync_debug": "false", 2019-12-22T22:28:30.847 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_sync_max_payload_keys": "2000", 2019-12-22T22:28:30.847 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_sync_max_payload_size": "1048576", 2019-12-22T22:28:30.847 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_sync_provider_kill_at": "0", 2019-12-22T22:28:30.847 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_sync_requester_kill_at": "0", 2019-12-22T22:28:30.847 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_sync_timeout": "60.000000", 2019-12-22T22:28:30.847 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_target_pg_per_osd": "100", 2019-12-22T22:28:30.847 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_tick_interval": "5", 2019-12-22T22:28:30.848 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_timecheck_interval": "300.000000", 2019-12-22T22:28:30.848 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_timecheck_skew_interval": "30.000000", 2019-12-22T22:28:30.848 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2019-12-22T22:28:30.848 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2019-12-22T22:28:30.848 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2019-12-22T22:28:30.848 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_warn_on_misplaced": "false", 2019-12-22T22:28:30.848 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_warn_on_msgr2_not_enabled": "false", 2019-12-22T22:28:30.849 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2019-12-22T22:28:30.849 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_warn_on_pool_no_app": "true", 2019-12-22T22:28:30.849 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_warn_on_pool_no_redundancy": "true", 2019-12-22T22:28:30.849 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2019-12-22T22:28:30.849 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2019-12-22T22:28:30.849 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2019-12-22T22:28:30.849 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_warn_on_too_few_osds": "false", 2019-12-22T22:28:30.849 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2019-12-22T22:28:30.850 INFO:teuthology.orchestra.run.smithi091.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2019-12-22T22:28:30.850 INFO:teuthology.orchestra.run.smithi091.stdout: "monmap": "", 2019-12-22T22:28:30.850 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_async_max_op_threads": "5", 2019-12-22T22:28:30.850 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_async_op_threads": "3", 2019-12-22T22:28:30.850 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_async_rdma_buffer_size": "131072", 2019-12-22T22:28:30.850 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_async_rdma_cm": "false", 2019-12-22T22:28:30.850 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_async_rdma_device_name": "", 2019-12-22T22:28:30.851 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_async_rdma_dscp": "96", 2019-12-22T22:28:30.851 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_async_rdma_enable_hugepage": "false", 2019-12-22T22:28:30.851 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_async_rdma_gid_idx": "0", 2019-12-22T22:28:30.851 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_async_rdma_local_gid": "", 2019-12-22T22:28:30.851 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_async_rdma_polling_us": "1000", 2019-12-22T22:28:30.851 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_async_rdma_port_num": "1", 2019-12-22T22:28:30.851 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_async_rdma_receive_buffers": "32768", 2019-12-22T22:28:30.852 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_async_rdma_receive_queue_len": "4096", 2019-12-22T22:28:30.852 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_async_rdma_roce_ver": "1", 2019-12-22T22:28:30.852 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_async_rdma_send_buffers": "1024", 2019-12-22T22:28:30.852 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_async_rdma_sl": "3", 2019-12-22T22:28:30.852 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_async_rdma_support_srq": "true", 2019-12-22T22:28:30.852 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_async_rdma_type": "ib", 2019-12-22T22:28:30.852 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_bind_before_connect": "false", 2019-12-22T22:28:30.852 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_bind_ipv4": "true", 2019-12-22T22:28:30.853 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_bind_ipv6": "false", 2019-12-22T22:28:30.853 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_bind_msgr1": "true", 2019-12-22T22:28:30.853 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_bind_msgr2": "true", 2019-12-22T22:28:30.853 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_bind_port_max": "7300", 2019-12-22T22:28:30.853 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_bind_port_min": "6800", 2019-12-22T22:28:30.853 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_bind_prefer_ipv4": "false", 2019-12-22T22:28:30.853 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_bind_retry_count": "3", 2019-12-22T22:28:30.854 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_bind_retry_delay": "5", 2019-12-22T22:28:30.854 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_blackhole_client": "false", 2019-12-22T22:28:30.854 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_blackhole_mds": "false", 2019-12-22T22:28:30.854 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_blackhole_mgr": "false", 2019-12-22T22:28:30.854 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_blackhole_mon": "false", 2019-12-22T22:28:30.854 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_blackhole_osd": "false", 2019-12-22T22:28:30.854 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_client_mode": "crc secure", 2019-12-22T22:28:30.855 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_cluster_mode": "crc secure", 2019-12-22T22:28:30.855 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_cluster_type": "", 2019-12-22T22:28:30.855 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_connection_idle_timeout": "900", 2019-12-22T22:28:30.855 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_connection_ready_timeout": "10", 2019-12-22T22:28:30.855 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_crc_data": "true", 2019-12-22T22:28:30.855 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_crc_header": "true", 2019-12-22T22:28:30.855 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_die_on_bad_msg": "false", 2019-12-22T22:28:30.856 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_die_on_bug": "true", 2019-12-22T22:28:30.856 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_die_on_old_message": "true", 2019-12-22T22:28:30.856 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_die_on_skipped_message": "false", 2019-12-22T22:28:30.856 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_die_on_unhandled_msg": "false", 2019-12-22T22:28:30.856 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_dispatch_throttle_bytes": "104857600", 2019-12-22T22:28:30.856 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_dpdk_coremask": "0xF", 2019-12-22T22:28:30.856 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_dpdk_debug_allow_loopback": "false", 2019-12-22T22:28:30.856 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2019-12-22T22:28:30.857 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_dpdk_host_ipv4_addr": "", 2019-12-22T22:28:30.857 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_dpdk_hugepages": "", 2019-12-22T22:28:30.857 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_dpdk_hw_flow_control": "true", 2019-12-22T22:28:30.857 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2019-12-22T22:28:30.858 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_dpdk_lro": "true", 2019-12-22T22:28:30.858 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_dpdk_memory_channel": "4", 2019-12-22T22:28:30.858 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2019-12-22T22:28:30.858 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_dpdk_pmd": "", 2019-12-22T22:28:30.858 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_dpdk_port_id": "0", 2019-12-22T22:28:30.858 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2019-12-22T22:28:30.859 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_dump_corrupt_message_level": "1", 2019-12-22T22:28:30.859 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_dump_on_send": "false", 2019-12-22T22:28:30.859 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_initial_backoff": "0.200000", 2019-12-22T22:28:30.859 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_inject_delay_max": "1.000000", 2019-12-22T22:28:30.859 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_inject_delay_msg_type": "", 2019-12-22T22:28:30.859 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_inject_delay_probability": "0.005000", 2019-12-22T22:28:30.860 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_inject_delay_type": "osd", 2019-12-22T22:28:30.860 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_inject_internal_delays": "0.002000", 2019-12-22T22:28:30.860 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_inject_socket_failures": "2500", 2019-12-22T22:28:30.860 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_learn_addr_from_peer": "true", 2019-12-22T22:28:30.860 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_max_accept_failures": "4", 2019-12-22T22:28:30.860 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_max_backoff": "15.000000", 2019-12-22T22:28:30.860 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_mon_client_mode": "secure crc", 2019-12-22T22:28:30.861 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_mon_cluster_mode": "secure crc", 2019-12-22T22:28:30.861 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_mon_service_mode": "secure crc", 2019-12-22T22:28:30.861 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2019-12-22T22:28:30.861 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_pq_min_cost": "65536", 2019-12-22T22:28:30.861 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_public_type": "", 2019-12-22T22:28:30.861 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_service_mode": "crc secure", 2019-12-22T22:28:30.861 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_tcp_listen_backlog": "512", 2019-12-22T22:28:30.862 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_tcp_nodelay": "true", 2019-12-22T22:28:30.862 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_tcp_prefetch_max_size": "4096", 2019-12-22T22:28:30.862 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_tcp_rcvbuf": "0", 2019-12-22T22:28:30.862 INFO:teuthology.orchestra.run.smithi091.stdout: "ms_type": "async+posix", 2019-12-22T22:28:30.862 INFO:teuthology.orchestra.run.smithi091.stdout: "no_config_file": "false", 2019-12-22T22:28:30.862 INFO:teuthology.orchestra.run.smithi091.stdout: "objecter_completion_locks_per_session": "32", 2019-12-22T22:28:30.862 INFO:teuthology.orchestra.run.smithi091.stdout: "objecter_debug_inject_relock_delay": "false", 2019-12-22T22:28:30.863 INFO:teuthology.orchestra.run.smithi091.stdout: "objecter_inflight_op_bytes": "104857600", 2019-12-22T22:28:30.863 INFO:teuthology.orchestra.run.smithi091.stdout: "objecter_inflight_ops": "1024", 2019-12-22T22:28:30.863 INFO:teuthology.orchestra.run.smithi091.stdout: "objecter_inject_no_watch_ping": "false", 2019-12-22T22:28:30.863 INFO:teuthology.orchestra.run.smithi091.stdout: "objecter_retry_writes_after_first_reply": "false", 2019-12-22T22:28:30.863 INFO:teuthology.orchestra.run.smithi091.stdout: "objecter_tick_interval": "5.000000", 2019-12-22T22:28:30.863 INFO:teuthology.orchestra.run.smithi091.stdout: "objecter_timeout": "10.000000", 2019-12-22T22:28:30.863 INFO:teuthology.orchestra.run.smithi091.stdout: "objectstore_blackhole": "false", 2019-12-22T22:28:30.863 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_agent_delay_time": "5.000000", 2019-12-22T22:28:30.864 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_agent_hist_halflife": "1000", 2019-12-22T22:28:30.864 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_agent_max_low_ops": "2", 2019-12-22T22:28:30.864 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_agent_max_ops": "4", 2019-12-22T22:28:30.864 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_agent_min_evict_effort": "0.100000", 2019-12-22T22:28:30.864 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_agent_quantize_effort": "0.100000", 2019-12-22T22:28:30.864 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_agent_slop": "0.020000", 2019-12-22T22:28:30.864 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_allow_recovery_below_min_size": "true", 2019-12-22T22:28:30.865 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_async_recovery_min_cost": "100", 2019-12-22T22:28:30.865 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_auto_mark_unfound_lost": "false", 2019-12-22T22:28:30.865 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_backfill_retry_interval": "30.000000", 2019-12-22T22:28:30.865 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_backfill_scan_max": "512", 2019-12-22T22:28:30.865 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_backfill_scan_min": "64", 2019-12-22T22:28:30.865 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_backoff_on_degraded": "false", 2019-12-22T22:28:30.865 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_backoff_on_peering": "false", 2019-12-22T22:28:30.866 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_backoff_on_unfound": "true", 2019-12-22T22:28:30.866 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_beacon_report_interval": "300", 2019-12-22T22:28:30.866 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_bench_duration": "30", 2019-12-22T22:28:30.866 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_bench_large_size_max_throughput": "104857600", 2019-12-22T22:28:30.866 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_bench_max_block_size": "67108864", 2019-12-22T22:28:30.866 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_bench_small_size_max_iops": "100", 2019-12-22T22:28:30.867 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_blkin_trace_all": "false", 2019-12-22T22:28:30.867 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2019-12-22T22:28:30.867 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2019-12-22T22:28:30.867 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_check_for_log_corruption": "false", 2019-12-22T22:28:30.867 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_check_max_object_name_len_on_startup": "true", 2019-12-22T22:28:30.868 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas", 2019-12-22T22:28:30.868 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2019-12-22T22:28:30.868 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas", 2019-12-22T22:28:30.869 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_class_update_on_start": "true", 2019-12-22T22:28:30.869 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_client_message_cap": "100", 2019-12-22T22:28:30.869 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_client_message_size_cap": "524288000", 2019-12-22T22:28:30.869 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_client_op_priority": "63", 2019-12-22T22:28:30.869 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_client_watch_timeout": "30", 2019-12-22T22:28:30.870 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_command_max_records": "256", 2019-12-22T22:28:30.870 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_command_thread_suicide_timeout": "900", 2019-12-22T22:28:30.870 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_command_thread_timeout": "600", 2019-12-22T22:28:30.870 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_copyfrom_max_chunk": "8388608", 2019-12-22T22:28:30.870 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_crush_chooseleaf_type": "1", 2019-12-22T22:28:30.870 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_crush_initial_weight": "-1.000000", 2019-12-22T22:28:30.870 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_crush_update_on_start": "true", 2019-12-22T22:28:30.871 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_crush_update_weight_set": "true", 2019-12-22T22:28:30.871 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2019-12-22T22:28:30.871 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2019-12-22T22:28:30.871 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2019-12-22T22:28:30.871 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_drop_ping_duration": "0", 2019-12-22T22:28:30.871 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_drop_ping_probability": "0.000000", 2019-12-22T22:28:30.871 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_feed_pullee": "-1", 2019-12-22T22:28:30.872 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_inject_copyfrom_error": "false", 2019-12-22T22:28:30.872 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2019-12-22T22:28:30.872 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2019-12-22T22:28:30.872 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_misdirected_ops": "false", 2019-12-22T22:28:30.872 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_no_acting_change": "false", 2019-12-22T22:28:30.872 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_no_purge_strays": "false", 2019-12-22T22:28:30.872 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_op_order": "false", 2019-12-22T22:28:30.873 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_pg_log_writeout": "false", 2019-12-22T22:28:30.873 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_pretend_recovery_active": "false", 2019-12-22T22:28:30.873 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_random_push_read_error": "0.000000", 2019-12-22T22:28:30.873 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2019-12-22T22:28:30.873 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_shutdown": "false", 2019-12-22T22:28:30.873 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2019-12-22T22:28:30.874 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2019-12-22T22:28:30.874 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_verify_cached_snaps": "false", 2019-12-22T22:28:30.874 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_verify_missing_on_start": "false", 2019-12-22T22:28:30.874 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_verify_snaps": "false", 2019-12-22T22:28:30.874 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_debug_verify_stray_on_activate": "false", 2019-12-22T22:28:30.874 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_deep_scrub_interval": "604800.000000", 2019-12-22T22:28:30.874 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_deep_scrub_keys": "1024", 2019-12-22T22:28:30.875 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2019-12-22T22:28:30.875 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2019-12-22T22:28:30.875 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2019-12-22T22:28:30.875 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_deep_scrub_stride": "524288", 2019-12-22T22:28:30.875 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2019-12-22T22:28:30.875 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_default_data_pool_replay_window": "45", 2019-12-22T22:28:30.875 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_default_notify_timeout": "30", 2019-12-22T22:28:30.876 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_delete_sleep": "0.000000", 2019-12-22T22:28:30.876 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_delete_sleep_hdd": "5.000000", 2019-12-22T22:28:30.876 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_delete_sleep_hybrid": "2.000000", 2019-12-22T22:28:30.876 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_delete_sleep_ssd": "0.000000", 2019-12-22T22:28:30.876 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_discard_disconnected_ops": "true", 2019-12-22T22:28:30.876 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_enable_op_tracker": "true", 2019-12-22T22:28:30.876 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_erasure_code_plugins": "jerasure lrc isa", 2019-12-22T22:28:30.877 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_failsafe_full_ratio": "0.970000", 2019-12-22T22:28:30.877 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_fast_fail_on_connection_refused": "true", 2019-12-22T22:28:30.877 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_fast_info": "true", 2019-12-22T22:28:30.877 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_fast_shutdown": "true", 2019-12-22T22:28:30.877 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_find_best_info_ignore_history_les": "false", 2019-12-22T22:28:30.877 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_force_auth_primary_missing_objects": "100", 2019-12-22T22:28:30.877 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2019-12-22T22:28:30.878 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_function_tracing": "false", 2019-12-22T22:28:30.878 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_heartbeat_grace": "20", 2019-12-22T22:28:30.878 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_heartbeat_interval": "6", 2019-12-22T22:28:30.878 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2019-12-22T22:28:30.878 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_heartbeat_min_peers": "10", 2019-12-22T22:28:30.878 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_heartbeat_min_size": "2000", 2019-12-22T22:28:30.878 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_heartbeat_stale": "600", 2019-12-22T22:28:30.878 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2019-12-22T22:28:30.879 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_hit_set_max_size": "100000", 2019-12-22T22:28:30.879 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_hit_set_min_size": "1000", 2019-12-22T22:28:30.879 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_hit_set_namespace": ".ceph-internal", 2019-12-22T22:28:30.879 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_ignore_stale_divergent_priors": "false", 2019-12-22T22:28:30.879 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2019-12-22T22:28:30.879 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_inject_failure_on_pg_removal": "false", 2019-12-22T22:28:30.879 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2019-12-22T22:28:30.880 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_journal_flush_on_shutdown": "true", 2019-12-22T22:28:30.880 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_journal_size": "5120", 2019-12-22T22:28:30.880 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_kill_backfill_at": "0", 2019-12-22T22:28:30.880 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_loop_before_reset_tphandle": "64", 2019-12-22T22:28:30.880 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_map_cache_size": "50", 2019-12-22T22:28:30.880 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_map_dedup": "true", 2019-12-22T22:28:30.880 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_map_message_max": "40", 2019-12-22T22:28:30.881 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_map_message_max_bytes": "10485760", 2019-12-22T22:28:30.881 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_map_share_max_epochs": "40", 2019-12-22T22:28:30.881 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_attr_name_len": "100", 2019-12-22T22:28:30.881 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_attr_size": "0", 2019-12-22T22:28:30.881 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_backfills": "1", 2019-12-22T22:28:30.881 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_markdown_count": "5", 2019-12-22T22:28:30.881 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_markdown_period": "600", 2019-12-22T22:28:30.881 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_object_name_len": "2048", 2019-12-22T22:28:30.882 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_object_namespace_len": "256", 2019-12-22T22:28:30.882 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_object_size": "134217728", 2019-12-22T22:28:30.882 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2019-12-22T22:28:30.882 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_omap_entries_per_request": "1024", 2019-12-22T22:28:30.882 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_pg_blocked_by": "16", 2019-12-22T22:28:30.882 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_pg_log_entries": "3000", 2019-12-22T22:28:30.882 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2019-12-22T22:28:30.883 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_pgls": "1024", 2019-12-22T22:28:30.883 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_push_cost": "8388608", 2019-12-22T22:28:30.883 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_push_objects": "10", 2019-12-22T22:28:30.883 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_scrubs": "1", 2019-12-22T22:28:30.883 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2019-12-22T22:28:30.883 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_trimming_pgs": "2", 2019-12-22T22:28:30.883 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_write_op_reply_len": "32", 2019-12-22T22:28:30.884 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_max_write_size": "90", 2019-12-22T22:28:30.884 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2019-12-22T22:28:30.884 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_mclock_scheduler_background_best_effort_lim": "999999", 2019-12-22T22:28:30.884 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_mclock_scheduler_background_best_effort_res": "1", 2019-12-22T22:28:30.884 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2019-12-22T22:28:30.884 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_mclock_scheduler_background_recovery_lim": "999999", 2019-12-22T22:28:30.884 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_mclock_scheduler_background_recovery_res": "1", 2019-12-22T22:28:30.885 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2019-12-22T22:28:30.885 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_mclock_scheduler_client_lim": "999999", 2019-12-22T22:28:30.885 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_mclock_scheduler_client_res": "1", 2019-12-22T22:28:30.885 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_mclock_scheduler_client_wgt": "1", 2019-12-22T22:28:30.885 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_memory_base": "805306368", 2019-12-22T22:28:30.885 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_memory_cache_min": "134217728", 2019-12-22T22:28:30.885 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_memory_cache_resize_interval": "1.000000", 2019-12-22T22:28:30.885 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_memory_expected_fragmentation": "0.150000", 2019-12-22T22:28:30.886 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_memory_target": "4294967296", 2019-12-22T22:28:30.886 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2019-12-22T22:28:30.886 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_min_pg_log_entries": "3000", 2019-12-22T22:28:30.886 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_min_recovery_priority": "0", 2019-12-22T22:28:30.886 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_mon_ack_timeout": "30.000000", 2019-12-22T22:28:30.886 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_mon_heartbeat_interval": "30", 2019-12-22T22:28:30.886 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2019-12-22T22:28:30.887 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_mon_report_interval": "5", 2019-12-22T22:28:30.887 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_mon_report_max_in_flight": "2", 2019-12-22T22:28:30.887 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_mon_shutdown_timeout": "5.000000", 2019-12-22T22:28:30.887 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_num_cache_shards": "32", 2019-12-22T22:28:30.887 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_num_op_tracker_shard": "32", 2019-12-22T22:28:30.887 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_numa_auto_affinity": "true", 2019-12-22T22:28:30.887 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_numa_node": "-1", 2019-12-22T22:28:30.888 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_numa_prefer_iface": "true", 2019-12-22T22:28:30.888 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_object_clean_region_max_num_intervals": "10", 2019-12-22T22:28:30.888 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_objecter_finishers": "1", 2019-12-22T22:28:30.888 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_objectstore": "bluestore", 2019-12-22T22:28:30.888 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_objectstore_fuse": "false", 2019-12-22T22:28:30.888 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_objectstore_tracing": "false", 2019-12-22T22:28:30.888 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_op_complaint_time": "30.000000", 2019-12-22T22:28:30.889 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_op_history_duration": "600", 2019-12-22T22:28:30.889 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_op_history_size": "20", 2019-12-22T22:28:30.889 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_op_history_slow_op_size": "20", 2019-12-22T22:28:30.889 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2019-12-22T22:28:30.889 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_op_log_threshold": "5", 2019-12-22T22:28:30.889 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_op_num_shards": "0", 2019-12-22T22:28:30.889 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_op_num_shards_hdd": "5", 2019-12-22T22:28:30.890 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_op_num_shards_ssd": "8", 2019-12-22T22:28:30.890 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_op_num_threads_per_shard": "0", 2019-12-22T22:28:30.890 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2019-12-22T22:28:30.890 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2019-12-22T22:28:30.890 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2019-12-22T22:28:30.891 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_op_pq_min_cost": "65536", 2019-12-22T22:28:30.891 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_op_queue": "wpq", 2019-12-22T22:28:30.891 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_op_queue_cut_off": "high", 2019-12-22T22:28:30.891 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_op_thread_suicide_timeout": "150", 2019-12-22T22:28:30.891 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_op_thread_timeout": "15", 2019-12-22T22:28:30.892 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_open_classes_on_start": "true", 2019-12-22T22:28:30.892 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_os_flags": "0", 2019-12-22T22:28:30.892 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_peering_op_priority": "255", 2019-12-22T22:28:30.892 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pg_delete_cost": "1048576", 2019-12-22T22:28:30.892 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pg_delete_priority": "5", 2019-12-22T22:28:30.893 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2019-12-22T22:28:30.893 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2019-12-22T22:28:30.893 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pg_log_dups_tracked": "3000", 2019-12-22T22:28:30.893 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pg_log_trim_max": "10000", 2019-12-22T22:28:30.893 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pg_log_trim_min": "100", 2019-12-22T22:28:30.894 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2019-12-22T22:28:30.894 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pg_object_context_cache_count": "64", 2019-12-22T22:28:30.894 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pg_stat_report_interval_max": "500", 2019-12-22T22:28:30.894 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2019-12-22T22:28:30.894 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_cache_min_evict_age": "0", 2019-12-22T22:28:30.895 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_cache_min_flush_age": "0", 2019-12-22T22:28:30.895 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2019-12-22T22:28:30.895 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2019-12-22T22:28:30.895 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2019-12-22T22:28:30.896 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_crush_rule": "-1", 2019-12-22T22:28:30.896 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_ec_fast_read": "false", 2019-12-22T22:28:30.896 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd", 2019-12-22T22:28:30.896 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_flag_hashpspool": "true", 2019-12-22T22:28:30.896 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_flag_nodelete": "false", 2019-12-22T22:28:30.897 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_flag_nopgchange": "false", 2019-12-22T22:28:30.897 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_flag_nosizechange": "false", 2019-12-22T22:28:30.897 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_flags": "0", 2019-12-22T22:28:30.897 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2019-12-22T22:28:30.898 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_min_size": "2", 2019-12-22T22:28:30.898 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2019-12-22T22:28:30.898 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_pg_num": "16", 2019-12-22T22:28:30.898 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_pgp_num": "0", 2019-12-22T22:28:30.899 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2019-12-22T22:28:30.899 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_size": "2", 2019-12-22T22:28:30.899 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_default_type": "replicated", 2019-12-22T22:28:30.899 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2019-12-22T22:28:30.899 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_pool_use_gmt_hitset": "true", 2019-12-22T22:28:30.899 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_push_per_object_cost": "1000", 2019-12-22T22:28:30.900 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_read_ec_check_for_errors": "false", 2019-12-22T22:28:30.900 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_recover_clone_overlap": "true", 2019-12-22T22:28:30.900 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_recover_clone_overlap_limit": "10", 2019-12-22T22:28:30.901 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_recovery_cost": "20971520", 2019-12-22T22:28:30.901 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_recovery_delay_start": "0.000000", 2019-12-22T22:28:30.901 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_recovery_max_active": "0", 2019-12-22T22:28:30.901 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_recovery_max_active_hdd": "3", 2019-12-22T22:28:30.901 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_recovery_max_active_ssd": "10", 2019-12-22T22:28:30.902 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_recovery_max_chunk": "8388608", 2019-12-22T22:28:30.902 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2019-12-22T22:28:30.902 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_recovery_max_single_start": "1", 2019-12-22T22:28:30.902 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_recovery_op_priority": "3", 2019-12-22T22:28:30.902 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_recovery_op_warn_multiple": "16", 2019-12-22T22:28:30.903 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_recovery_priority": "5", 2019-12-22T22:28:30.903 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_recovery_retry_interval": "30.000000", 2019-12-22T22:28:30.903 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_recovery_sleep": "0.000000", 2019-12-22T22:28:30.903 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_recovery_sleep_hdd": "0.100000", 2019-12-22T22:28:30.904 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2019-12-22T22:28:30.904 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_recovery_sleep_ssd": "0.000000", 2019-12-22T22:28:30.904 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_repair_during_recovery": "false", 2019-12-22T22:28:30.904 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_requested_scrub_priority": "120", 2019-12-22T22:28:30.904 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_rollback_to_cluster_snap": "", 2019-12-22T22:28:30.905 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_auto_repair": "false", 2019-12-22T22:28:30.905 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_auto_repair_num_errors": "5", 2019-12-22T22:28:30.905 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_backoff_ratio": "0.660000", 2019-12-22T22:28:30.905 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_begin_hour": "0", 2019-12-22T22:28:30.906 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_begin_week_day": "0", 2019-12-22T22:28:30.906 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_chunk_max": "25", 2019-12-22T22:28:30.906 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_chunk_min": "5", 2019-12-22T22:28:30.906 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_cost": "52428800", 2019-12-22T22:28:30.906 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_during_recovery": "false", 2019-12-22T22:28:30.907 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_end_hour": "24", 2019-12-22T22:28:30.907 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_end_week_day": "7", 2019-12-22T22:28:30.907 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_extended_sleep": "0.000000", 2019-12-22T22:28:30.907 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2019-12-22T22:28:30.908 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_invalid_stats": "true", 2019-12-22T22:28:30.908 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_load_threshold": "0.500000", 2019-12-22T22:28:30.908 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_max_interval": "604800.000000", 2019-12-22T22:28:30.908 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_max_preemptions": "5", 2019-12-22T22:28:30.909 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_min_interval": "86400.000000", 2019-12-22T22:28:30.909 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_priority": "5", 2019-12-22T22:28:30.909 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_scrub_sleep": "0.000000", 2019-12-22T22:28:30.909 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_shutdown_pgref_assert": "false", 2019-12-22T22:28:30.909 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_skip_data_digest": "false", 2019-12-22T22:28:30.910 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_smart_report_timeout": "5", 2019-12-22T22:28:30.910 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_snap_trim_cost": "1048576", 2019-12-22T22:28:30.910 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_snap_trim_priority": "5", 2019-12-22T22:28:30.910 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_snap_trim_sleep": "0.000000", 2019-12-22T22:28:30.910 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2019-12-22T22:28:30.910 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2019-12-22T22:28:30.910 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2019-12-22T22:28:30.910 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_stats_ack_timeout_decay": "0.900000", 2019-12-22T22:28:30.911 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_stats_ack_timeout_factor": "2.000000", 2019-12-22T22:28:30.911 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_target_transaction_size": "30", 2019-12-22T22:28:30.911 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_tier_default_cache_hit_set_count": "4", 2019-12-22T22:28:30.911 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2019-12-22T22:28:30.911 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2019-12-22T22:28:30.912 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2019-12-22T22:28:30.912 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2019-12-22T22:28:30.912 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2019-12-22T22:28:30.912 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2019-12-22T22:28:30.912 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_tier_default_cache_mode": "writeback", 2019-12-22T22:28:30.912 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2019-12-22T22:28:30.913 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_tier_promote_max_objects_sec": "25", 2019-12-22T22:28:30.913 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_tracing": "false", 2019-12-22T22:28:30.913 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_use_stale_snap": "false", 2019-12-22T22:28:30.913 INFO:teuthology.orchestra.run.smithi091.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2019-12-22T22:28:30.914 INFO:teuthology.orchestra.run.smithi091.stdout: "osdc_blkin_trace_all": "false", 2019-12-22T22:28:30.914 INFO:teuthology.orchestra.run.smithi091.stdout: "paxos_kill_at": "0", 2019-12-22T22:28:30.914 INFO:teuthology.orchestra.run.smithi091.stdout: "paxos_max_join_drift": "10", 2019-12-22T22:28:30.914 INFO:teuthology.orchestra.run.smithi091.stdout: "paxos_min": "500", 2019-12-22T22:28:30.914 INFO:teuthology.orchestra.run.smithi091.stdout: "paxos_min_wait": "0.050000", 2019-12-22T22:28:30.914 INFO:teuthology.orchestra.run.smithi091.stdout: "paxos_propose_interval": "1.000000", 2019-12-22T22:28:30.914 INFO:teuthology.orchestra.run.smithi091.stdout: "paxos_service_trim_max": "500", 2019-12-22T22:28:30.915 INFO:teuthology.orchestra.run.smithi091.stdout: "paxos_service_trim_min": "10", 2019-12-22T22:28:30.915 INFO:teuthology.orchestra.run.smithi091.stdout: "paxos_stash_full_interval": "25", 2019-12-22T22:28:30.915 INFO:teuthology.orchestra.run.smithi091.stdout: "paxos_trim_max": "500", 2019-12-22T22:28:30.915 INFO:teuthology.orchestra.run.smithi091.stdout: "paxos_trim_min": "250", 2019-12-22T22:28:30.915 INFO:teuthology.orchestra.run.smithi091.stdout: "perf": "true", 2019-12-22T22:28:30.915 INFO:teuthology.orchestra.run.smithi091.stdout: "pid_file": "", 2019-12-22T22:28:30.915 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin_crypto_accelerator": "crypto_isal", 2019-12-22T22:28:30.915 INFO:teuthology.orchestra.run.smithi091.stdout: "plugin_dir": "/usr/lib64/ceph", 2019-12-22T22:28:30.916 INFO:teuthology.orchestra.run.smithi091.stdout: "prompt": "\u001b[01;33mCephFS:~\u001b[96m/\u001b[0m\u001b[01;33m>>>\u001b[00m ", 2019-12-22T22:28:30.916 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addr": "-", 2019-12-22T22:28:30.916 INFO:teuthology.orchestra.run.smithi091.stdout: "public_addrv": "", 2019-12-22T22:28:30.916 INFO:teuthology.orchestra.run.smithi091.stdout: "public_bind_addr": "-", 2019-12-22T22:28:30.916 INFO:teuthology.orchestra.run.smithi091.stdout: "public_network": "", 2019-12-22T22:28:30.916 INFO:teuthology.orchestra.run.smithi091.stdout: "public_network_interface": "", 2019-12-22T22:28:30.916 INFO:teuthology.orchestra.run.smithi091.stdout: "qat_compressor_enabled": "false", 2019-12-22T22:28:30.917 INFO:teuthology.orchestra.run.smithi091.stdout: "quiet": "false", 2019-12-22T22:28:30.917 INFO:teuthology.orchestra.run.smithi091.stdout: "rados_mon_op_timeout": "0.000000", 2019-12-22T22:28:30.917 INFO:teuthology.orchestra.run.smithi091.stdout: "rados_osd_op_timeout": "0.000000", 2019-12-22T22:28:30.917 INFO:teuthology.orchestra.run.smithi091.stdout: "rados_tracing": "false", 2019-12-22T22:28:30.917 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_atime_update_interval": "60", 2019-12-22T22:28:30.917 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2019-12-22T22:28:30.917 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_balance_parent_reads": "false", 2019-12-22T22:28:30.918 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_balance_snap_reads": "false", 2019-12-22T22:28:30.918 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_blacklist_expire_seconds": "0", 2019-12-22T22:28:30.918 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_blacklist_on_break_lock": "true", 2019-12-22T22:28:30.918 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_blkin_trace_all": "false", 2019-12-22T22:28:30.918 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_cache": "true", 2019-12-22T22:28:30.918 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_cache_block_writes_upfront": "false", 2019-12-22T22:28:30.918 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_cache_max_dirty": "25165824", 2019-12-22T22:28:30.919 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_cache_max_dirty_age": "1.000000", 2019-12-22T22:28:30.919 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_cache_max_dirty_object": "0", 2019-12-22T22:28:30.919 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_cache_policy": "writearound", 2019-12-22T22:28:30.919 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_cache_size": "33554432", 2019-12-22T22:28:30.919 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_cache_target_dirty": "16777216", 2019-12-22T22:28:30.919 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_cache_writethrough_until_flush": "true", 2019-12-22T22:28:30.919 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_clone_copy_on_read": "false", 2019-12-22T22:28:30.919 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_concurrent_management_ops": "10", 2019-12-22T22:28:30.920 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_default_clone_format": "auto", 2019-12-22T22:28:30.920 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_default_data_pool": "", 2019-12-22T22:28:30.920 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_default_features": "61", 2019-12-22T22:28:30.920 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_default_format": "2", 2019-12-22T22:28:30.920 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_default_map_options": "", 2019-12-22T22:28:30.920 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_default_order": "22", 2019-12-22T22:28:30.920 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_default_pool": "rbd", 2019-12-22T22:28:30.921 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_default_stripe_count": "0", 2019-12-22T22:28:30.921 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_default_stripe_unit": "0", 2019-12-22T22:28:30.921 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_disable_zero_copy_writes": "true", 2019-12-22T22:28:30.921 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_discard_granularity_bytes": "65536", 2019-12-22T22:28:30.921 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_discard_on_zeroed_write_same": "true", 2019-12-22T22:28:30.921 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_enable_alloc_hint": "true", 2019-12-22T22:28:30.921 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_io_scheduler": "simple", 2019-12-22T22:28:30.922 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2019-12-22T22:28:30.922 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_journal_commit_age": "5.000000", 2019-12-22T22:28:30.922 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2019-12-22T22:28:30.922 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_journal_max_payload_bytes": "16384", 2019-12-22T22:28:30.922 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_journal_object_flush_age": "0.000000", 2019-12-22T22:28:30.922 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_journal_object_flush_bytes": "1048576", 2019-12-22T22:28:30.922 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_journal_object_flush_interval": "0", 2019-12-22T22:28:30.923 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2019-12-22T22:28:30.923 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2019-12-22T22:28:30.923 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_journal_order": "24", 2019-12-22T22:28:30.923 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_journal_pool": "", 2019-12-22T22:28:30.923 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_journal_splay_width": "4", 2019-12-22T22:28:30.923 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_localize_parent_reads": "false", 2019-12-22T22:28:30.923 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_localize_snap_reads": "false", 2019-12-22T22:28:30.923 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2019-12-22T22:28:30.924 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2019-12-22T22:28:30.924 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2019-12-22T22:28:30.924 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2019-12-22T22:28:30.924 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2019-12-22T22:28:30.924 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2019-12-22T22:28:30.924 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_image_policy_type": "simple", 2019-12-22T22:28:30.924 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2019-12-22T22:28:30.925 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_image_state_check_interval": "30", 2019-12-22T22:28:30.925 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2019-12-22T22:28:30.925 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2019-12-22T22:28:30.925 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2019-12-22T22:28:30.925 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2019-12-22T22:28:30.925 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2019-12-22T22:28:30.925 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_memory_autotune": "true", 2019-12-22T22:28:30.926 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_memory_base": "805306368", 2019-12-22T22:28:30.926 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2019-12-22T22:28:30.926 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_memory_cache_min": "134217728", 2019-12-22T22:28:30.926 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2019-12-22T22:28:30.926 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2019-12-22T22:28:30.926 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_memory_target": "4294967296", 2019-12-22T22:28:30.926 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_perf_stats_prio": "5", 2019-12-22T22:28:30.927 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2019-12-22T22:28:30.927 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2019-12-22T22:28:30.927 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirroring_delete_delay": "0", 2019-12-22T22:28:30.927 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirroring_max_mirroring_snapshots": "3", 2019-12-22T22:28:30.927 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirroring_replay_delay": "0", 2019-12-22T22:28:30.927 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2019-12-22T22:28:30.927 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2019-12-22T22:28:30.927 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_move_to_trash_on_remove": "false", 2019-12-22T22:28:30.928 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2019-12-22T22:28:30.928 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_mtime_update_interval": "60", 2019-12-22T22:28:30.928 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_non_blocking_aio": "true", 2019-12-22T22:28:30.928 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_op_thread_timeout": "60", 2019-12-22T22:28:30.928 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_op_threads": "1", 2019-12-22T22:28:30.928 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_parent_cache_enabled": "false", 2019-12-22T22:28:30.928 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_qos_bps_burst": "0", 2019-12-22T22:28:30.929 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_qos_bps_limit": "0", 2019-12-22T22:28:30.929 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_qos_iops_burst": "0", 2019-12-22T22:28:30.929 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_qos_iops_limit": "0", 2019-12-22T22:28:30.929 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_qos_read_bps_burst": "0", 2019-12-22T22:28:30.929 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_qos_read_bps_limit": "0", 2019-12-22T22:28:30.929 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_qos_read_iops_burst": "0", 2019-12-22T22:28:30.929 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_qos_read_iops_limit": "0", 2019-12-22T22:28:30.930 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_qos_schedule_tick_min": "50", 2019-12-22T22:28:30.930 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_qos_write_bps_burst": "0", 2019-12-22T22:28:30.930 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_qos_write_bps_limit": "0", 2019-12-22T22:28:30.930 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_qos_write_iops_burst": "0", 2019-12-22T22:28:30.930 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_qos_write_iops_limit": "0", 2019-12-22T22:28:30.930 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2019-12-22T22:28:30.930 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_readahead_max_bytes": "524288", 2019-12-22T22:28:30.931 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_readahead_trigger_requests": "10", 2019-12-22T22:28:30.931 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_request_timed_out_seconds": "30", 2019-12-22T22:28:30.931 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_rwl_enabled": "false", 2019-12-22T22:28:30.931 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_rwl_log_periodic_stats": "false", 2019-12-22T22:28:30.931 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_rwl_path": "/tmp", 2019-12-22T22:28:30.931 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_rwl_size": "1073741824", 2019-12-22T22:28:30.931 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_skip_partial_discard": "true", 2019-12-22T22:28:30.932 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2019-12-22T22:28:30.932 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_tracing": "false", 2019-12-22T22:28:30.932 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_validate_names": "true", 2019-12-22T22:28:30.932 INFO:teuthology.orchestra.run.smithi091.stdout: "rbd_validate_pool": "true", 2019-12-22T22:28:30.932 INFO:teuthology.orchestra.run.smithi091.stdout: "restapi_base_url": "", 2019-12-22T22:28:30.932 INFO:teuthology.orchestra.run.smithi091.stdout: "restapi_log_level": "", 2019-12-22T22:28:30.932 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_acl_grants_max_num": "100", 2019-12-22T22:28:30.933 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_admin_entry": "admin", 2019-12-22T22:28:30.933 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_barbican_url": "", 2019-12-22T22:28:30.933 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2019-12-22T22:28:30.933 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_bucket_default_quota_max_size": "-1", 2019-12-22T22:28:30.933 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_bucket_index_max_aio": "128", 2019-12-22T22:28:30.933 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_bucket_quota_cache_size": "10000", 2019-12-22T22:28:30.933 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_bucket_quota_soft_threshold": "0.950000", 2019-12-22T22:28:30.934 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_bucket_quota_ttl": "600", 2019-12-22T22:28:30.934 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_cache_enabled": "true", 2019-12-22T22:28:30.934 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_cache_expiry_interval": "900", 2019-12-22T22:28:30.934 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_cache_lru_size": "10000", 2019-12-22T22:28:30.934 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_content_length_compat": "false", 2019-12-22T22:28:30.934 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_copy_obj_progress": "true", 2019-12-22T22:28:30.934 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2019-12-22T22:28:30.934 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_cors_rules_max_num": "100", 2019-12-22T22:28:30.935 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_cross_domain_policy": "", 2019-12-22T22:28:30.935 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_crypt_default_encryption_key": "", 2019-12-22T22:28:30.935 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_crypt_require_ssl": "true", 2019-12-22T22:28:30.935 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2019-12-22T22:28:30.935 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2019-12-22T22:28:30.935 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_crypt_suppress_logs": "true", 2019-12-22T22:28:30.935 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_crypt_vault_addr": "", 2019-12-22T22:28:30.936 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_crypt_vault_auth": "token", 2019-12-22T22:28:30.936 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_crypt_vault_namespace": "", 2019-12-22T22:28:30.936 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_crypt_vault_prefix": "", 2019-12-22T22:28:30.936 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_crypt_vault_secret_engine": "transit", 2019-12-22T22:28:30.936 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_crypt_vault_token_file": "", 2019-12-22T22:28:30.936 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_curl_low_speed_limit": "1024", 2019-12-22T22:28:30.936 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_curl_low_speed_time": "300", 2019-12-22T22:28:30.937 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_curl_wait_timeout_ms": "1000", 2019-12-22T22:28:30.937 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2019-12-22T22:28:30.937 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_data_log_changes_size": "1000", 2019-12-22T22:28:30.937 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_data_log_num_shards": "128", 2019-12-22T22:28:30.937 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_data_log_obj_prefix": "data_log", 2019-12-22T22:28:30.937 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_data_log_window": "30", 2019-12-22T22:28:30.937 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_data_notify_interval_msec": "200", 2019-12-22T22:28:30.938 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_default_realm_info_oid": "default.realm", 2019-12-22T22:28:30.938 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_default_region_info_oid": "default.region", 2019-12-22T22:28:30.938 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_default_zone_info_oid": "default.zone", 2019-12-22T22:28:30.938 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2019-12-22T22:28:30.938 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_defer_to_bucket_acls": "", 2019-12-22T22:28:30.938 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_delete_multi_obj_max_num": "1000", 2019-12-22T22:28:30.938 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_dmclock_admin_lim": "0.000000", 2019-12-22T22:28:30.939 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_dmclock_admin_res": "100.000000", 2019-12-22T22:28:30.939 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2019-12-22T22:28:30.939 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_dmclock_auth_lim": "0.000000", 2019-12-22T22:28:30.939 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_dmclock_auth_res": "200.000000", 2019-12-22T22:28:30.939 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2019-12-22T22:28:30.939 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_dmclock_data_lim": "0.000000", 2019-12-22T22:28:30.939 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_dmclock_data_res": "500.000000", 2019-12-22T22:28:30.939 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_dmclock_data_wgt": "500.000000", 2019-12-22T22:28:30.940 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2019-12-22T22:28:30.940 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_dmclock_metadata_res": "500.000000", 2019-12-22T22:28:30.940 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2019-12-22T22:28:30.940 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_dns_name": "", 2019-12-22T22:28:30.940 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_dns_s3website_name": "", 2019-12-22T22:28:30.940 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_dynamic_resharding": "true", 2019-12-22T22:28:30.940 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, pubsub", 2019-12-22T22:28:30.941 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_enable_gc_threads": "true", 2019-12-22T22:28:30.941 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_enable_lc_threads": "true", 2019-12-22T22:28:30.941 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_enable_ops_log": "false", 2019-12-22T22:28:30.941 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_enable_quota_threads": "true", 2019-12-22T22:28:30.941 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_enable_static_website": "false", 2019-12-22T22:28:30.941 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_enable_usage_log": "false", 2019-12-22T22:28:30.941 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_enforce_swift_acls": "true", 2019-12-22T22:28:30.942 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_exit_timeout_secs": "120", 2019-12-22T22:28:30.942 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_expose_bucket": "false", 2019-12-22T22:28:30.942 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_extended_http_attrs": "", 2019-12-22T22:28:30.942 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_fcgi_socket_backlog": "1024", 2019-12-22T22:28:30.942 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_frontends": "beast port=7480", 2019-12-22T22:28:30.942 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_gc_max_concurrent_io": "10", 2019-12-22T22:28:30.942 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_gc_max_deferred": "50", 2019-12-22T22:28:30.942 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2019-12-22T22:28:30.943 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_gc_max_objs": "32", 2019-12-22T22:28:30.943 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_gc_max_queue_size": "134213632", 2019-12-22T22:28:30.943 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_gc_max_trim_chunk": "16", 2019-12-22T22:28:30.943 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_gc_obj_min_wait": "7200", 2019-12-22T22:28:30.943 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_gc_processor_max_time": "3600", 2019-12-22T22:28:30.943 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_gc_processor_period": "3600", 2019-12-22T22:28:30.943 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_get_obj_max_req_size": "4194304", 2019-12-22T22:28:30.944 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_get_obj_window_size": "16777216", 2019-12-22T22:28:30.944 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_healthcheck_disabling_path": "", 2019-12-22T22:28:30.944 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_host": "", 2019-12-22T22:28:30.944 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_ignore_get_invalid_range": "false", 2019-12-22T22:28:30.944 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_init_timeout": "300", 2019-12-22T22:28:30.944 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2019-12-22T22:28:30.944 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_accepted_admin_roles": "", 2019-12-22T22:28:30.945 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2019-12-22T22:28:30.945 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_admin_domain": "", 2019-12-22T22:28:30.945 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_admin_password": "", 2019-12-22T22:28:30.945 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_admin_password_path": "", 2019-12-22T22:28:30.945 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_admin_project": "", 2019-12-22T22:28:30.945 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_admin_tenant": "", 2019-12-22T22:28:30.945 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_admin_token": "", 2019-12-22T22:28:30.946 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_admin_token_path": "", 2019-12-22T22:28:30.946 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_admin_user": "", 2019-12-22T22:28:30.946 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_api_version": "2", 2019-12-22T22:28:30.946 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_barbican_domain": "", 2019-12-22T22:28:30.946 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_barbican_password": "", 2019-12-22T22:28:30.946 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_barbican_project": "", 2019-12-22T22:28:30.946 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_barbican_tenant": "", 2019-12-22T22:28:30.947 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_barbican_user": "", 2019-12-22T22:28:30.947 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_implicit_tenants": "false", 2019-12-22T22:28:30.947 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_token_cache_size": "10000", 2019-12-22T22:28:30.947 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_url": "", 2019-12-22T22:28:30.947 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_keystone_verify_ssl": "true", 2019-12-22T22:28:30.947 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_lc_debug_interval": "-1", 2019-12-22T22:28:30.947 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_lc_lock_max_time": "60", 2019-12-22T22:28:30.947 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_lc_max_objs": "32", 2019-12-22T22:28:30.948 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_lc_max_rules": "1000", 2019-12-22T22:28:30.948 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_lc_thread_delay": "0", 2019-12-22T22:28:30.948 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2019-12-22T22:28:30.948 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_ldap_dnattr": "uid", 2019-12-22T22:28:30.948 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2019-12-22T22:28:30.948 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_ldap_searchfilter": "", 2019-12-22T22:28:30.948 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2019-12-22T22:28:30.949 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_ldap_uri": "ldaps://", 2019-12-22T22:28:30.949 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2019-12-22T22:28:30.949 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_list_bucket_min_readahead": "1000", 2019-12-22T22:28:30.949 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_list_buckets_max_chunk": "1000", 2019-12-22T22:28:30.949 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_log_http_headers": "", 2019-12-22T22:28:30.949 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_log_nonexistent_bucket": "false", 2019-12-22T22:28:30.950 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2019-12-22T22:28:30.950 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_log_object_name_utc": "false", 2019-12-22T22:28:30.950 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_max_attr_name_len": "0", 2019-12-22T22:28:30.950 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_max_attr_size": "0", 2019-12-22T22:28:30.951 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_max_attrs_num_in_req": "0", 2019-12-22T22:28:30.951 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_max_chunk_size": "4194304", 2019-12-22T22:28:30.951 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_max_concurrent_requests": "1024", 2019-12-22T22:28:30.952 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_max_dynamic_shards": "1999", 2019-12-22T22:28:30.952 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_max_listing_results": "1000", 2019-12-22T22:28:30.952 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_max_notify_retries": "3", 2019-12-22T22:28:30.952 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_max_objs_per_shard": "100000", 2019-12-22T22:28:30.952 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_max_put_param_size": "1048576", 2019-12-22T22:28:30.953 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_max_put_size": "5368709120", 2019-12-22T22:28:30.953 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_max_slo_entries": "1000", 2019-12-22T22:28:30.954 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_md_log_max_shards": "64", 2019-12-22T22:28:30.954 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_md_notify_interval_msec": "200", 2019-12-22T22:28:30.954 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_mime_types_file": "/etc/mime.types", 2019-12-22T22:28:30.954 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_mp_lock_max_time": "600", 2019-12-22T22:28:30.955 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_multipart_min_part_size": "5242880", 2019-12-22T22:28:30.955 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_multipart_part_upload_limit": "10000", 2019-12-22T22:28:30.955 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_nfs_fhcache_partitions": "3", 2019-12-22T22:28:30.955 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_nfs_fhcache_size": "2017", 2019-12-22T22:28:30.955 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2019-12-22T22:28:30.955 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_nfs_lru_lanes": "5", 2019-12-22T22:28:30.956 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_nfs_max_gc": "300", 2019-12-22T22:28:30.956 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_nfs_namespace_expire_secs": "300", 2019-12-22T22:28:30.956 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_nfs_s3_fast_attrs": "false", 2019-12-22T22:28:30.956 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_nfs_write_completion_interval_s": "10", 2019-12-22T22:28:30.956 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_num_async_rados_threads": "32", 2019-12-22T22:28:30.956 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_num_control_oids": "8", 2019-12-22T22:28:30.956 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_num_rados_handles": "1", 2019-12-22T22:28:30.957 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_obj_stripe_size": "4194304", 2019-12-22T22:28:30.957 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_obj_tombstone_cache_size": "1000", 2019-12-22T22:28:30.957 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_objexp_chunk_size": "100", 2019-12-22T22:28:30.957 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_objexp_gc_interval": "600", 2019-12-22T22:28:30.957 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_objexp_hints_num_shards": "127", 2019-12-22T22:28:30.957 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_olh_pending_timeout_sec": "3600", 2019-12-22T22:28:30.957 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_op_thread_suicide_timeout": "0", 2019-12-22T22:28:30.957 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_op_thread_timeout": "600", 2019-12-22T22:28:30.958 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_op_tracing": "false", 2019-12-22T22:28:30.958 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_opa_token": "", 2019-12-22T22:28:30.958 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_opa_url": "", 2019-12-22T22:28:30.958 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_opa_verify_ssl": "true", 2019-12-22T22:28:30.958 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_ops_log_data_backlog": "5242880", 2019-12-22T22:28:30.958 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_ops_log_rados": "true", 2019-12-22T22:28:30.958 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_ops_log_socket_path": "", 2019-12-22T22:28:30.959 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_override_bucket_index_max_shards": "0", 2019-12-22T22:28:30.959 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2019-12-22T22:28:30.959 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_period_push_interval": "2.000000", 2019-12-22T22:28:30.959 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_period_push_interval_max": "30.000000", 2019-12-22T22:28:30.959 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_period_root_pool": ".rgw.root", 2019-12-22T22:28:30.959 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_port": "", 2019-12-22T22:28:30.960 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_print_continue": "true", 2019-12-22T22:28:30.960 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_print_prohibited_content_length": "false", 2019-12-22T22:28:30.960 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_put_obj_max_window_size": "67108864", 2019-12-22T22:28:30.960 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_put_obj_min_window_size": "16777216", 2019-12-22T22:28:30.960 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2019-12-22T22:28:30.961 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_rados_pool_pg_num_min": "8", 2019-12-22T22:28:30.961 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_rados_pool_recovery_priority": "5", 2019-12-22T22:28:30.961 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_rados_tracing": "false", 2019-12-22T22:28:30.961 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_realm": "", 2019-12-22T22:28:30.961 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_realm_root_pool": ".rgw.root", 2019-12-22T22:28:30.961 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_region": "", 2019-12-22T22:28:30.961 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_region_root_pool": ".rgw.root", 2019-12-22T22:28:30.962 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_relaxed_region_enforcement": "false", 2019-12-22T22:28:30.962 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_relaxed_s3_bucket_names": "false", 2019-12-22T22:28:30.962 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2019-12-22T22:28:30.962 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_request_uri": "", 2019-12-22T22:28:30.962 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_reshard_batch_size": "64", 2019-12-22T22:28:30.962 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_reshard_bucket_lock_duration": "360", 2019-12-22T22:28:30.962 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_reshard_max_aio": "128", 2019-12-22T22:28:30.963 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_reshard_num_logs": "16", 2019-12-22T22:28:30.963 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_reshard_thread_interval": "600", 2019-12-22T22:28:30.963 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_resolve_cname": "false", 2019-12-22T22:28:30.963 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_rest_getusage_op_compat": "false", 2019-12-22T22:28:30.963 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_run_sync_thread": "true", 2019-12-22T22:28:30.963 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_s3_auth_order": "sts, external, local", 2019-12-22T22:28:30.963 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_s3_auth_use_keystone": "false", 2019-12-22T22:28:30.963 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_s3_auth_use_ldap": "false", 2019-12-22T22:28:30.964 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_s3_auth_use_rados": "true", 2019-12-22T22:28:30.964 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_s3_auth_use_sts": "false", 2019-12-22T22:28:30.964 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_s3_success_create_obj_status": "0", 2019-12-22T22:28:30.964 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_safe_max_objects_per_shard": "102400", 2019-12-22T22:28:30.964 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_scheduler_type": "throttler", 2019-12-22T22:28:30.964 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_script_uri": "", 2019-12-22T22:28:30.964 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_service_provider_name": "", 2019-12-22T22:28:30.965 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_shard_warning_threshold": "90.000000", 2019-12-22T22:28:30.965 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_socket_path": "", 2019-12-22T22:28:30.965 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_sts_client_id": "", 2019-12-22T22:28:30.965 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_sts_client_secret": "", 2019-12-22T22:28:30.965 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_sts_entry": "sts", 2019-12-22T22:28:30.965 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_sts_key": "sts", 2019-12-22T22:28:30.965 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_sts_max_session_duration": "43200", 2019-12-22T22:28:30.966 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_sts_token_introspection_url": "", 2019-12-22T22:28:30.966 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_swift_account_in_url": "false", 2019-12-22T22:28:30.966 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_swift_auth_entry": "auth", 2019-12-22T22:28:30.966 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_swift_auth_url": "", 2019-12-22T22:28:30.966 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_swift_custom_header": "", 2019-12-22T22:28:30.966 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_swift_enforce_content_length": "false", 2019-12-22T22:28:30.966 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_swift_need_stats": "true", 2019-12-22T22:28:30.967 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_swift_tenant_name": "", 2019-12-22T22:28:30.967 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_swift_token_expiration": "86400", 2019-12-22T22:28:30.967 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_swift_url": "", 2019-12-22T22:28:30.967 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_swift_url_prefix": "swift", 2019-12-22T22:28:30.967 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_swift_versioning_enabled": "false", 2019-12-22T22:28:30.967 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2019-12-22T22:28:30.968 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_sync_lease_period": "120", 2019-12-22T22:28:30.968 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2019-12-22T22:28:30.968 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_sync_log_trim_interval": "1200", 2019-12-22T22:28:30.968 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_sync_log_trim_max_buckets": "16", 2019-12-22T22:28:30.968 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2019-12-22T22:28:30.968 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2019-12-22T22:28:30.968 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_sync_trace_history_size": "4096", 2019-12-22T22:28:30.969 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_sync_trace_per_node_log_size": "32", 2019-12-22T22:28:30.969 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2019-12-22T22:28:30.969 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_thread_pool_size": "512", 2019-12-22T22:28:30.969 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_torrent_comment": "", 2019-12-22T22:28:30.969 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_torrent_createby": "", 2019-12-22T22:28:30.969 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_torrent_encoding": "", 2019-12-22T22:28:30.969 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_torrent_flag": "false", 2019-12-22T22:28:30.970 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_torrent_origin": "", 2019-12-22T22:28:30.970 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_torrent_sha_unit": "524288", 2019-12-22T22:28:30.970 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_torrent_tracker": "", 2019-12-22T22:28:30.970 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_trust_forwarded_https": "false", 2019-12-22T22:28:30.970 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_usage_log_flush_threshold": "1024", 2019-12-22T22:28:30.970 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_usage_log_tick_interval": "30", 2019-12-22T22:28:30.970 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_usage_max_shards": "32", 2019-12-22T22:28:30.971 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_usage_max_user_shards": "1", 2019-12-22T22:28:30.971 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_use_opa_authz": "false", 2019-12-22T22:28:30.971 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_user_default_quota_max_objects": "-1", 2019-12-22T22:28:30.971 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_user_default_quota_max_size": "-1", 2019-12-22T22:28:30.971 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_user_max_buckets": "1000", 2019-12-22T22:28:30.971 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2019-12-22T22:28:30.971 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_user_quota_sync_idle_users": "false", 2019-12-22T22:28:30.971 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_user_quota_sync_interval": "86400", 2019-12-22T22:28:30.972 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_user_quota_sync_wait_time": "86400", 2019-12-22T22:28:30.972 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_user_unique_email": "true", 2019-12-22T22:28:30.972 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_verify_ssl": "true", 2019-12-22T22:28:30.972 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_website_routing_rules_max_num": "50", 2019-12-22T22:28:30.972 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_zone": "", 2019-12-22T22:28:30.972 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_zone_root_pool": ".rgw.root", 2019-12-22T22:28:30.973 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_zonegroup": "", 2019-12-22T22:28:30.973 INFO:teuthology.orchestra.run.smithi091.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2019-12-22T22:28:30.973 INFO:teuthology.orchestra.run.smithi091.stdout: "rocksdb_block_size": "4096", 2019-12-22T22:28:30.973 INFO:teuthology.orchestra.run.smithi091.stdout: "rocksdb_bloom_bits_per_key": "20", 2019-12-22T22:28:30.973 INFO:teuthology.orchestra.run.smithi091.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2019-12-22T22:28:30.973 INFO:teuthology.orchestra.run.smithi091.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "true", 2019-12-22T22:28:30.974 INFO:teuthology.orchestra.run.smithi091.stdout: "rocksdb_cache_row_ratio": "0.000000", 2019-12-22T22:28:30.975 INFO:teuthology.orchestra.run.smithi091.stdout: "rocksdb_cache_shard_bits": "4", 2019-12-22T22:28:30.975 INFO:teuthology.orchestra.run.smithi091.stdout: "rocksdb_cache_size": "536870912", 2019-12-22T22:28:30.975 INFO:teuthology.orchestra.run.smithi091.stdout: "rocksdb_cache_type": "binned_lru", 2019-12-22T22:28:30.975 INFO:teuthology.orchestra.run.smithi091.stdout: "rocksdb_collect_compaction_stats": "false", 2019-12-22T22:28:30.975 INFO:teuthology.orchestra.run.smithi091.stdout: "rocksdb_collect_extended_stats": "false", 2019-12-22T22:28:30.976 INFO:teuthology.orchestra.run.smithi091.stdout: "rocksdb_collect_memory_stats": "false", 2019-12-22T22:28:30.976 INFO:teuthology.orchestra.run.smithi091.stdout: "rocksdb_delete_range_threshold": "1024", 2019-12-22T22:28:30.976 INFO:teuthology.orchestra.run.smithi091.stdout: "rocksdb_index_type": "binary_search", 2019-12-22T22:28:30.976 INFO:teuthology.orchestra.run.smithi091.stdout: "rocksdb_log_to_ceph_log": "true", 2019-12-22T22:28:30.976 INFO:teuthology.orchestra.run.smithi091.stdout: "rocksdb_metadata_block_size": "4096", 2019-12-22T22:28:30.976 INFO:teuthology.orchestra.run.smithi091.stdout: "rocksdb_partition_filters": "false", 2019-12-22T22:28:30.977 INFO:teuthology.orchestra.run.smithi091.stdout: "rocksdb_perf": "false", 2019-12-22T22:28:30.977 INFO:teuthology.orchestra.run.smithi091.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2019-12-22T22:28:30.977 INFO:teuthology.orchestra.run.smithi091.stdout: "rotating_keys_bootstrap_timeout": "30", 2019-12-22T22:28:30.977 INFO:teuthology.orchestra.run.smithi091.stdout: "rotating_keys_renewal_timeout": "10", 2019-12-22T22:28:30.977 INFO:teuthology.orchestra.run.smithi091.stdout: "run_dir": "/var/run/ceph", 2019-12-22T22:28:30.978 INFO:teuthology.orchestra.run.smithi091.stdout: "setgroup": "ceph", 2019-12-22T22:28:30.978 INFO:teuthology.orchestra.run.smithi091.stdout: "setuser": "ceph", 2019-12-22T22:28:30.978 INFO:teuthology.orchestra.run.smithi091.stdout: "setuser_match_path": "", 2019-12-22T22:28:30.978 INFO:teuthology.orchestra.run.smithi091.stdout: "target_max_misplaced_ratio": "0.050000", 2019-12-22T22:28:30.978 INFO:teuthology.orchestra.run.smithi091.stdout: "thp": "false", 2019-12-22T22:28:30.978 INFO:teuthology.orchestra.run.smithi091.stdout: "threadpool_default_timeout": "60", 2019-12-22T22:28:30.978 INFO:teuthology.orchestra.run.smithi091.stdout: "threadpool_empty_queue_max_wait": "2", 2019-12-22T22:28:30.979 INFO:teuthology.orchestra.run.smithi091.stdout: "throttler_perf_counter": "true", 2019-12-22T22:28:30.979 INFO:teuthology.orchestra.run.smithi091.stdout: "timing": "false" 2019-12-22T22:28:30.979 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:28:31.216 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2019-12-22T22:28:33.262 INFO:teuthology.orchestra.run.smithi091.stdout:mon.a: {} 2019-12-22T22:28:33.289 INFO:teuthology.orchestra.run.smithi091.stdout:mon.c: {} 2019-12-22T22:28:33.304 INFO:teuthology.orchestra.run.smithi091.stdout:mon.b: {} 2019-12-22T22:28:33.304 INFO:teuthology.orchestra.run.smithi091.stderr:mon.a: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2019-12-22T22:28:33.304 INFO:teuthology.orchestra.run.smithi091.stderr:mon.c: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2019-12-22T22:28:33.305 INFO:teuthology.orchestra.run.smithi091.stderr:mon.b: mon_osd_down_out_interval = '0' (not observed, change may require restart) 2019-12-22T22:28:33.846 INFO:teuthology.orchestra.run.smithi091:> type ceph-objectstore-tool 2019-12-22T22:28:33.965 INFO:teuthology.orchestra.run.smithi091.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2019-12-22T22:28:33.966 INFO:teuthology.orchestra.run.smithi043:> type ceph-objectstore-tool 2019-12-22T22:28:34.075 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2019-12-22T22:28:34.076 INFO:teuthology.orchestra.run.smithi107:> type ceph-objectstore-tool 2019-12-22T22:28:34.166 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2019-12-22T22:28:34.166 INFO:teuthology.run_tasks:Running task sleep... 2019-12-22T22:28:34.239 INFO:teuthology.task.sleep:Sleeping for 600000 seconds 2019-12-22T22:28:34.239 INFO:tasks.thrashosds.thrasher:starting do_thrash 2019-12-22T22:28:34.244 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11] out_osds: [] dead_osds: [] live_osds: [11, 10, 1, 0, 3, 2, 5, 4, 7, 6, 9, 8] 2019-12-22T22:28:34.245 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 2019-12-22T22:28:34.245 INFO:tasks.thrashosds.thrasher:Killing osd 9, live_osds are [11, 10, 1, 0, 3, 2, 5, 4, 7, 6, 9, 8] 2019-12-22T22:28:34.245 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2019-12-22T22:28:34.245 INFO:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-147630da-2509-11ea-8284-001a4aab830c@osd.9 2019-12-22T22:28:34.246 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2019-12-22T22:28:34.250 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last): File "/home/teuthworker/src/github.com_liewegas_ceph_wip-old-clients-cephadm/qa/tasks/ceph_manager.py", line 992, in wrapper return func(self) File "/home/teuthworker/src/github.com_liewegas_ceph_wip-old-clients-cephadm/qa/tasks/ceph_manager.py", line 1009, in do_sighup self.ceph_manager.signal_osd(osd, signal.SIGHUP, silent=True) File "/home/teuthworker/src/github.com_liewegas_ceph_wip-old-clients-cephadm/qa/tasks/ceph_manager.py", line 2707, in signal_osd self.cluster).signal(sig, silent=silent) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/daemon/cephadmunit.py", line 95, in signal raise NotImplementedError NotImplementedError 2019-12-22T22:28:34.251 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "src/gevent/greenlet.py", line 766, in gevent._greenlet.Greenlet.run File "/home/teuthworker/src/github.com_liewegas_ceph_wip-old-clients-cephadm/qa/tasks/ceph_manager.py", line 992, in wrapper return func(self) File "/home/teuthworker/src/github.com_liewegas_ceph_wip-old-clients-cephadm/qa/tasks/ceph_manager.py", line 1009, in do_sighup self.ceph_manager.signal_osd(osd, signal.SIGHUP, silent=True) File "/home/teuthworker/src/github.com_liewegas_ceph_wip-old-clients-cephadm/qa/tasks/ceph_manager.py", line 2707, in signal_osd self.cluster).signal(sig, silent=silent) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/daemon/cephadmunit.py", line 95, in signal raise NotImplementedError NotImplementedError 2019-12-22T22:28:34.252 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2019-12-22T22:28:34.252 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:28:34.252 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2019-12-22T22:28:34.253 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T22:28:34.253 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2019-12-22T22:28:34.253 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:28:34.696 INFO:ceph.osd.9.smithi043.stdout:2019-12-22T22:28:34.464+0000 7f8636973700 -1 received signal: Terminated from Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm() ) UID: 0 2019-12-22T22:28:34.697 INFO:ceph.osd.9.smithi043.stdout:2019-12-22T22:28:34.464+0000 7f8636973700 -1 osd.9 74 *** Got signal Terminated *** 2019-12-22T22:28:34.697 INFO:ceph.osd.9.smithi043.stdout:2019-12-22T22:28:34.464+0000 7f8636973700 -1 osd.9 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2019-12-22T22:28:36.270 INFO:tasks.cephadm.osd.9:Stopped osd.9 2019-12-22T22:28:36.270 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd 2019-12-22T22:28:36.270 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last): File "/home/teuthworker/src/github.com_liewegas_ceph_wip-old-clients-cephadm/qa/tasks/ceph_manager.py", line 992, in wrapper return func(self) File "/home/teuthworker/src/github.com_liewegas_ceph_wip-old-clients-cephadm/qa/tasks/ceph_manager.py", line 1122, in _do_thrash self.choose_action()() File "/home/teuthworker/src/github.com_liewegas_ceph_wip-old-clients-cephadm/qa/tasks/ceph_manager.py", line 243, in kill_osd self.ceph_manager.ctx.ceph[self.cluster].conf['osd']): AttributeError: 'Namespace' object has no attribute 'conf' 2019-12-22T22:28:36.271 ERROR:tasks.thrashosds.thrasher:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_liewegas_ceph_wip-old-clients-cephadm/qa/tasks/ceph_manager.py", line 980, in do_thrash self._do_thrash() File "/home/teuthworker/src/github.com_liewegas_ceph_wip-old-clients-cephadm/qa/tasks/ceph_manager.py", line 992, in wrapper return func(self) File "/home/teuthworker/src/github.com_liewegas_ceph_wip-old-clients-cephadm/qa/tasks/ceph_manager.py", line 1122, in _do_thrash self.choose_action()() File "/home/teuthworker/src/github.com_liewegas_ceph_wip-old-clients-cephadm/qa/tasks/ceph_manager.py", line 243, in kill_osd self.ceph_manager.ctx.ceph[self.cluster].conf['osd']): AttributeError: 'Namespace' object has no attribute 'conf' 2019-12-22T22:28:37.554 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:28:37.756 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:28:38.129 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:28:38.130 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T22:28:38.600 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:28:38.652 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:28:38.697 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:28:38.738 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:28:38.779 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:28:38.824 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:28:38.866 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:28:38.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:28:38.940 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:28:38.973 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:28:38.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:28:38.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:28:38.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:28:38.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:28:38.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:28:38.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:28:38.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:28:38.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:28:38.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:28:38.976 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:28:38.979 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:28:38.979 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:28:38.979 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:28:39.002 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:28:40.178 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:28:40.178 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:28:40.755 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:28:41.305 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:28:41.306 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T22:28:41.751 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:28:42.179 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:28:44.280 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:28:45.157 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:28:45.157 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T22:28:45.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:28:45.565 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:28:45.597 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:28:45.624 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:28:45.652 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:28:45.679 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:28:45.706 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:28:45.733 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:28:45.760 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:28:45.807 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:28:45.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:28:45.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:28:45.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:28:45.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:28:45.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:28:45.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:28:45.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:28:45.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:28:45.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:28:45.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:28:45.850 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:28:45.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:28:45.851 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:28:45.851 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:28:45.984 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:28:46.611 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:28:46.611 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:28:47.838 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:28:48.395 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:28:48.395 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T22:28:48.612 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:28:48.932 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:28:51.408 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:28:51.933 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:28:51.961 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:28:51.988 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:28:52.014 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:28:52.018 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:28:52.019 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T22:28:52.041 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:28:52.073 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:28:52.095 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:28:52.119 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:28:52.146 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:28:52.172 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:28:52.197 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:28:52.206 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:28:52.206 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:28:52.206 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:28:52.206 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:28:52.206 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:28:52.206 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:28:52.207 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:28:52.207 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:28:52.207 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:28:52.207 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:28:52.207 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:28:52.207 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:28:52.207 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:28:52.870 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:28:52.870 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:28:52.885 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:28:54.535 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:28:54.871 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:28:55.035 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:28:55.035 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T22:28:55.668 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:28:57.824 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:28:58.167 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:28:58.167 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:28:58.167 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:28:58.167 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:28:58.694 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:28:58.741 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:28:58.776 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T22:28:58.784 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:28:58.820 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:28:58.856 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:28:58.881 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:28:58.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:28:58.934 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:28:58.961 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:28:58.987 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:28:58.988 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:28:58.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:28:58.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:28:58.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:28:58.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:28:58.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:28:58.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:28:58.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:28:58.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:28:58.990 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:28:58.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:28:58.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:28:58.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:28:59.672 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:28:59.673 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:28:59.999 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:29:01.437 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:29:01.437 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:29:01.437 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:29:01.437 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:29:01.438 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:29:01.674 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:29:02.029 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T22:29:02.780 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:29:04.856 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:29:04.903 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:29:04.955 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:29:04.995 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:29:05.038 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:29:05.039 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:29:05.069 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:29:05.095 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:29:05.132 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:29:05.185 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:29:05.233 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:29:05.269 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:29:05.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:29:05.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:29:05.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:29:05.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:29:05.273 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:29:05.273 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:29:05.273 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:29:05.273 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:29:05.273 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:29:05.273 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:29:05.273 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:29:05.274 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:29:05.274 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:29:05.831 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T22:29:06.003 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:29:06.004 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:29:06.900 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:29:08.008 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:29:08.166 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:29:08.818 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T22:29:09.547 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:29:10.491 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:29:11.352 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:29:11.387 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:29:11.417 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:29:11.449 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:29:11.479 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:29:11.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:29:11.560 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:29:11.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:29:11.651 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:29:11.683 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:29:11.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:29:11.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:29:11.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:29:11.686 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:29:11.686 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:29:11.686 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:29:11.686 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:29:11.687 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:29:11.687 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:29:11.687 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:29:11.688 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:29:11.688 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:29:11.688 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:29:11.696 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:29:11.696 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:29:11.696 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:29:11.697 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:29:11.697 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:29:12.268 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:29:12.268 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:29:12.619 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T22:29:13.996 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:29:14.270 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:29:14.928 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:29:14.928 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:29:14.928 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:29:14.928 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:29:14.929 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:29:15.503 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T22:29:16.630 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:29:16.903 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:29:16.925 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:29:16.948 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:29:16.977 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:29:17.005 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:29:17.031 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:29:17.056 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:29:17.083 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:29:17.109 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:29:17.136 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:29:17.163 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:29:17.164 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:29:17.164 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:29:17.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:29:17.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:29:17.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:29:17.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:29:17.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:29:17.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:29:17.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:29:17.166 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:29:17.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:29:17.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:29:17.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:29:18.246 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:29:18.246 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:29:18.438 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:29:19.089 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T22:29:20.034 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:29:20.247 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:29:21.631 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:29:22.365 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T22:29:22.825 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:29:23.041 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:29:23.891 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:29:23.945 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:29:23.980 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:29:24.014 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:29:24.044 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:29:24.070 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:29:24.096 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:29:24.123 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:29:24.150 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:29:24.181 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:29:24.184 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:29:24.184 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:29:24.184 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:29:24.184 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:29:24.184 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:29:24.184 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:29:24.185 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:29:24.185 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:29:24.185 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:29:24.185 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:29:24.185 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:29:24.185 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:29:24.948 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:29:24.949 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:29:25.218 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:29:25.218 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:29:25.219 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:29:25.219 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:29:25.219 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:29:25.790 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T22:29:26.887 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:29:26.954 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:29:28.482 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:29:28.483 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:29:28.483 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:29:28.483 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:29:29.205 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T22:29:29.571 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:29:29.776 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:29:30.628 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:29:30.674 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:29:30.734 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:29:30.736 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:29:30.769 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:29:30.792 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:29:30.819 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:29:30.860 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:29:30.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:29:30.944 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:29:30.945 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:29:30.946 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:29:30.946 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:29:30.946 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:29:30.946 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:29:30.946 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:29:30.946 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:29:30.947 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:29:30.947 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:29:30.947 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:29:30.947 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:29:30.947 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:29:30.947 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:29:31.859 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:29:31.859 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:29:32.275 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:29:32.905 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:29:32.923 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T22:29:33.861 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:29:35.283 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:29:35.523 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:29:36.036 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T22:29:36.771 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:29:36.792 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:29:36.816 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:29:36.839 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:29:36.867 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:29:36.901 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:29:36.922 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:29:36.951 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:29:36.977 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:29:37.000 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:29:37.026 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:29:37.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:29:37.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:29:37.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:29:37.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:29:37.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:29:37.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:29:37.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:29:37.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:29:37.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:29:37.029 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:29:37.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:29:37.030 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:29:37.030 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:29:37.876 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:29:37.877 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:29:38.814 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:29:39.299 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:29:39.300 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T22:29:39.878 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:29:39.935 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:29:42.073 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:29:42.524 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:29:42.785 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:29:42.810 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:29:42.833 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:29:42.860 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:29:42.861 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T22:29:42.865 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:29:42.892 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:29:42.921 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:29:42.950 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:29:42.974 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:29:43.000 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:29:43.030 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:29:43.056 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:29:43.057 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:29:43.058 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:29:43.058 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:29:43.058 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:29:43.058 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:29:43.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:29:43.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:29:43.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:29:43.060 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:29:43.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:29:43.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:29:43.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:29:43.973 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:29:43.974 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:29:45.489 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:29:45.971 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:29:45.975 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:29:45.984 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:29:45.984 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T22:29:48.490 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:29:48.629 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:29:48.674 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:29:48.699 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:29:48.725 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:29:48.751 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:29:48.779 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:29:48.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:29:48.830 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:29:48.857 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:29:48.859 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:29:48.884 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:29:48.910 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:29:48.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:29:48.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:29:48.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:29:48.913 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:29:48.913 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:29:48.913 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:29:48.913 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:29:48.913 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:29:48.913 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:29:48.913 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:29:48.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:29:48.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:29:48.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:29:49.667 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:29:49.668 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T22:29:49.902 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:29:49.902 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:29:51.902 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:29:51.904 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:29:52.070 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:29:52.909 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:29:52.909 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T22:29:54.569 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:29:54.661 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:29:54.684 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:29:54.712 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:29:54.737 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:29:54.766 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:29:54.791 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:29:54.830 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:29:54.845 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:29:54.872 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:29:54.901 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:29:54.929 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:29:54.929 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:29:54.929 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:29:54.929 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:29:54.929 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:29:54.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:29:54.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:29:54.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:29:54.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:29:54.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:29:54.930 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:29:54.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:29:54.931 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:29:54.931 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:29:55.681 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:29:55.801 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:29:55.802 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:29:56.281 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:29:56.281 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T22:29:57.803 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:29:58.546 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:29:58.938 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:29:59.405 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:29:59.405 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T22:30:00.754 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:30:00.776 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:30:00.801 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:30:00.823 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:30:00.849 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:30:00.873 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:30:00.900 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:30:00.923 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:30:00.949 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:30:00.977 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:30:01.003 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:30:01.004 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:30:01.005 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:30:01.005 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:30:01.005 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:30:01.005 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:30:01.006 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:30:01.006 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:30:01.006 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:30:01.006 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:30:01.006 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:30:01.006 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:30:01.006 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:30:01.007 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:30:01.667 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:30:01.667 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:30:01.914 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:30:02.010 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:30:02.537 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:30:02.538 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T22:30:03.667 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:30:04.960 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:30:05.123 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:30:06.341 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:30:06.342 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T22:30:06.839 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:30:07.710 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:30:07.762 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:30:07.804 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:30:07.862 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:30:07.901 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:30:07.935 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:30:07.967 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:30:07.998 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:30:08.027 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:30:08.032 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:30:08.054 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:30:08.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:30:08.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:30:08.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:30:08.056 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:30:08.056 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:30:08.056 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:30:08.056 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:30:08.057 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:30:08.057 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:30:08.058 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:30:08.058 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:30:08.058 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:30:08.059 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:30:08.679 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:30:08.692 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:30:08.692 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:30:09.327 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:30:09.328 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T22:30:10.693 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:30:11.939 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:30:12.284 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:30:13.257 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:30:13.258 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T22:30:13.469 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:30:14.322 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:30:14.375 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:30:14.426 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:30:14.461 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:30:14.488 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:30:14.518 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:30:14.549 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:30:14.574 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:30:14.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:30:14.637 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:30:14.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:30:14.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:30:14.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:30:14.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:30:14.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:30:14.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:30:14.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:30:14.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:30:14.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:30:14.640 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:30:14.641 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:30:14.641 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:30:14.645 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:30:14.798 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:30:15.347 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:30:15.348 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:30:15.709 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:30:16.480 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:30:16.480 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T22:30:17.349 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:30:18.014 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:30:19.664 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:30:20.229 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:30:20.254 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:30:20.254 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T22:30:20.255 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:30:20.279 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:30:20.306 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:30:20.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:30:20.360 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:30:20.408 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:30:20.457 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:30:20.501 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:30:20.528 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:30:20.556 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:30:20.557 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:30:20.557 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:30:20.558 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:30:20.558 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:30:20.558 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:30:20.558 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:30:20.558 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:30:20.558 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:30:20.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:30:20.559 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:30:20.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:30:20.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:30:20.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:30:20.751 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:30:21.206 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:30:21.206 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:30:23.055 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:30:23.207 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:30:23.576 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:30:23.577 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T22:30:23.927 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:30:25.925 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:30:25.950 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:30:25.973 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:30:25.997 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:30:26.023 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:30:26.049 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:30:26.077 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:30:26.102 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:30:26.129 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:30:26.156 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:30:26.183 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:30:26.184 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:30:26.185 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:30:26.185 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:30:26.185 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:30:26.185 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:30:26.186 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:30:26.186 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:30:26.186 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:30:26.186 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:30:26.187 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:30:26.187 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:30:26.187 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:30:26.187 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:30:26.683 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:30:26.785 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:30:26.785 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:30:26.974 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:30:27.247 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:30:27.247 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T22:30:28.787 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:30:29.941 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:30:30.000 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:30:31.155 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:30:31.156 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T22:30:31.837 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:30:32.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:30:32.734 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:30:32.765 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:30:32.800 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:30:32.840 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:30:32.865 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:30:32.895 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:30:33.026 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:30:33.026 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:30:33.027 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:30:33.027 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:30:33.027 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:30:33.027 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:30:33.027 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:30:33.027 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:30:33.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:30:33.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:30:33.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:30:33.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:30:33.028 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:30:33.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:30:33.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:30:33.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:30:33.160 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:30:33.224 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:30:33.742 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:30:33.743 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:30:34.040 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:30:34.040 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T22:30:35.744 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:30:36.349 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:30:37.005 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:30:37.006 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T22:30:37.007 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:30:38.870 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:30:38.894 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:30:38.916 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:30:38.940 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:30:38.968 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:30:38.999 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:30:39.020 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:30:39.046 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:30:39.073 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:30:39.105 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:30:39.129 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:30:39.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:30:39.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:30:39.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:30:39.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:30:39.130 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:30:39.130 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:30:39.130 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:30:39.130 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:30:39.130 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:30:39.130 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:30:39.130 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:30:39.131 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:30:39.131 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:30:39.717 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:30:39.747 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:30:39.748 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:30:39.919 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:30:40.175 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:30:40.175 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T22:30:41.748 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:30:42.785 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:30:42.928 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:30:43.821 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:30:43.821 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T22:30:44.562 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:30:44.584 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:30:44.625 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:30:44.653 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:30:44.680 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:30:44.708 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:30:44.734 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:30:44.761 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:30:44.787 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:30:44.815 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:30:44.840 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:30:44.842 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:30:44.842 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:30:44.843 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:30:44.843 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:30:44.843 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:30:44.843 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:30:44.843 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:30:44.843 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:30:44.843 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:30:44.844 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:30:44.844 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:30:44.844 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:30:44.844 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:30:45.677 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:30:45.677 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:30:46.117 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:30:46.223 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:30:46.761 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:30:46.761 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T22:30:47.678 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:30:49.748 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:30:50.007 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:30:50.554 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T22:30:50.959 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:30:50.983 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:30:51.004 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:30:51.027 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:30:51.053 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:30:51.079 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:30:51.105 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:30:51.130 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:30:51.159 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:30:51.203 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:30:51.251 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:30:51.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:30:51.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:30:51.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:30:51.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:30:51.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:30:51.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:30:51.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:30:51.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:30:51.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:30:51.255 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:30:51.255 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:30:51.255 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:30:51.255 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:30:51.986 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:30:51.987 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:30:52.910 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:30:52.922 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:30:52.922 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:30:52.922 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:30:52.922 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:30:52.922 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:30:53.458 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T22:30:53.993 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:30:56.009 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:30:56.265 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:30:56.265 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:30:56.266 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:30:56.266 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:30:56.266 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:30:57.185 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T22:30:57.594 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:30:57.619 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:30:57.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:30:57.663 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:30:57.694 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:30:57.717 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:30:57.740 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:30:57.767 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:30:57.793 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:30:57.824 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:30:57.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:30:57.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:30:57.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:30:57.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:30:57.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:30:57.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:30:57.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:30:57.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:30:57.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:30:57.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:30:57.850 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:30:57.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:30:57.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:30:57.851 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:30:58.444 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:30:58.445 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:30:58.624 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:30:59.526 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:31:00.139 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T22:31:00.447 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:31:02.642 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:31:02.935 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:31:03.465 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T22:31:03.883 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:31:03.916 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:31:03.948 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:31:03.978 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:31:04.006 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:31:04.033 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:31:04.060 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:31:04.090 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:31:04.116 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:31:04.143 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:31:04.180 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:31:04.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:31:04.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:31:04.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:31:04.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:31:04.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:31:04.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:31:04.191 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:31:04.191 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:31:04.191 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:31:04.192 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:31:04.192 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:31:04.192 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:31:04.192 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:31:04.903 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:31:04.903 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:31:05.794 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:31:05.794 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:31:05.794 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:31:05.794 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:31:05.794 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:31:05.798 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:31:06.421 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T22:31:06.904 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:31:08.958 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:31:09.648 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:31:09.649 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:31:09.649 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:31:09.649 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:31:10.268 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T22:31:10.468 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:31:10.502 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:31:10.527 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:31:10.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:31:10.578 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:31:10.605 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:31:10.631 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:31:10.659 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:31:10.702 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:31:10.745 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:31:10.777 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:31:10.779 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:31:10.779 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:31:10.779 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:31:10.780 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:31:10.780 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:31:10.780 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:31:10.780 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:31:10.780 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:31:10.781 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:31:10.781 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:31:10.781 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:31:10.782 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:31:10.782 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:31:11.373 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:31:11.373 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:31:11.917 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:31:12.383 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:31:13.118 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T22:31:13.374 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:31:14.930 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:31:16.208 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:31:16.706 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:31:16.729 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:31:16.749 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T22:31:16.757 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:31:16.780 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:31:16.807 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:31:16.833 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:31:16.860 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:31:16.886 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:31:16.925 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:31:16.941 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:31:16.968 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:31:16.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:31:16.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:31:16.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:31:16.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:31:16.970 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:31:16.970 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:31:16.970 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:31:16.970 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:31:16.970 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:31:16.971 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:31:16.971 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:31:16.971 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:31:16.971 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:31:17.647 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:31:17.647 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:31:17.734 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:31:19.418 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:31:19.418 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:31:19.418 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:31:19.418 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:31:19.649 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:31:20.088 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T22:31:20.555 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:31:22.620 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:31:22.647 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:31:22.672 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:31:22.695 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:31:22.722 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:31:22.749 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:31:22.774 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:31:22.779 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:31:22.818 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:31:22.854 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:31:22.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:31:22.923 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:31:22.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:31:22.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:31:22.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:31:22.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:31:22.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:31:22.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:31:22.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:31:22.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:31:22.928 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:31:22.928 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:31:22.928 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:31:22.928 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:31:22.928 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:31:23.250 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:31:23.562 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T22:31:23.728 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:31:23.728 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:31:25.729 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:31:26.158 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:31:26.952 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:31:26.963 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T22:31:28.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:31:28.638 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:31:28.663 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:31:28.688 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:31:28.714 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:31:28.741 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:31:28.767 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:31:28.794 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:31:28.825 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:31:28.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:31:28.874 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:31:28.876 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:31:28.876 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:31:28.876 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:31:28.876 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:31:28.876 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:31:28.877 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:31:28.877 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:31:28.877 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:31:28.877 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:31:28.877 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:31:28.877 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:31:28.877 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:31:28.878 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:31:29.654 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:31:29.654 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:31:29.854 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:31:29.860 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:31:30.442 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:31:30.442 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T22:31:31.655 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:31:32.941 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:31:33.692 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:31:34.506 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:31:34.506 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T22:31:34.920 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:31:34.943 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:31:34.966 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:31:34.990 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:31:35.016 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:31:35.042 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:31:35.069 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:31:35.095 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:31:35.121 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:31:35.148 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:31:35.177 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:31:35.177 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:31:35.177 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:31:35.177 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:31:35.177 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:31:35.178 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:31:35.178 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:31:35.178 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:31:35.178 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:31:35.178 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:31:35.178 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:31:35.178 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:31:35.179 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:31:35.179 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:31:35.747 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:31:35.748 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:31:35.940 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:31:36.675 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:31:37.233 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:31:37.234 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T22:31:37.749 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:31:38.970 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:31:40.077 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:31:40.805 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:31:40.805 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T22:31:41.002 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:31:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:31:41.063 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:31:41.084 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:31:41.111 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:31:41.136 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:31:41.162 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:31:41.190 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:31:41.216 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:31:41.244 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:31:41.270 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:31:41.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:31:41.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:31:41.273 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:31:41.273 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:31:41.273 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:31:41.273 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:31:41.273 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:31:41.273 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:31:41.273 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:31:41.274 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:31:41.274 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:31:41.274 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:31:41.274 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:31:41.700 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:31:41.849 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:31:41.850 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:31:43.057 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:31:43.819 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:31:43.819 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T22:31:43.851 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:31:44.954 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:31:46.756 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:31:46.783 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:31:46.816 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:31:46.845 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:31:46.884 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:31:46.896 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:31:46.922 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:31:46.949 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:31:46.967 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:31:46.983 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:31:47.015 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:31:47.046 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:31:47.047 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:31:47.048 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:31:47.048 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:31:47.048 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:31:47.048 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:31:47.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:31:47.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:31:47.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:31:47.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:31:47.050 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:31:47.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:31:47.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:31:47.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:31:47.715 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:31:47.716 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T22:31:47.876 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:31:47.877 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:31:47.961 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:31:49.878 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:31:50.215 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:31:51.128 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:31:51.129 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T22:31:51.954 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:31:52.621 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:31:52.646 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:31:52.670 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:31:52.696 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:31:52.721 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:31:52.748 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:31:52.774 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:31:52.812 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:31:52.830 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:31:52.860 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:31:52.883 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:31:52.885 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:31:52.886 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:31:52.886 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:31:52.887 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:31:52.887 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:31:52.887 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:31:52.887 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:31:52.887 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:31:52.888 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:31:52.888 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:31:52.888 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:31:52.888 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:31:52.889 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:31:53.572 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:31:53.572 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:31:53.940 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:31:54.465 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:31:54.465 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T22:31:55.064 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:31:55.573 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:31:57.491 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:31:58.295 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:31:58.296 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T22:31:58.757 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:31:59.025 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:31:59.611 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:31:59.658 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:31:59.708 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:31:59.738 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:31:59.769 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:31:59.796 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:31:59.821 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:31:59.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:31:59.874 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:31:59.900 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:31:59.901 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:31:59.901 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:31:59.901 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:31:59.902 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:31:59.902 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:31:59.902 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:31:59.902 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:31:59.902 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:31:59.903 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:31:59.903 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:31:59.903 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:31:59.903 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:31:59.903 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:32:00.564 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:32:00.564 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:32:01.167 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:32:01.718 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:32:01.727 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:32:01.728 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T22:32:02.565 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:32:04.565 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:32:05.300 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:32:05.303 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T22:32:05.903 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:32:05.929 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:32:05.946 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:32:05.961 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:32:05.985 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:32:06.007 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:32:06.041 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:32:06.071 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:32:06.098 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:32:06.132 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:32:06.158 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:32:06.185 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:32:06.186 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:32:06.186 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:32:06.187 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:32:06.187 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:32:06.187 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:32:06.187 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:32:06.187 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:32:06.188 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:32:06.188 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:32:06.188 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:32:06.188 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:32:06.189 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:32:06.189 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:32:07.212 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:32:07.212 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:32:07.736 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:32:08.235 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:32:08.235 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T22:32:09.208 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:32:09.214 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:32:11.233 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:32:12.081 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:32:12.082 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T22:32:12.251 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:32:12.964 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:32:13.111 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:32:13.152 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:32:13.191 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:32:13.225 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:32:13.258 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:32:13.313 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:32:13.353 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:32:13.392 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:32:13.425 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:32:13.452 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:32:13.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:32:13.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:32:13.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:32:13.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:32:13.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:32:13.455 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:32:13.455 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:32:13.455 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:32:13.455 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:32:13.455 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:32:13.455 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:32:13.456 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:32:13.456 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:32:14.241 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:32:14.242 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:32:14.691 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:32:15.209 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:32:15.209 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T22:32:15.976 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:32:16.244 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:32:17.909 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:32:18.856 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:32:18.856 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T22:32:19.600 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:32:19.983 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:32:20.452 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:32:20.498 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:32:20.537 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:32:20.571 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:32:20.601 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:32:20.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:32:20.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:32:20.730 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:32:20.792 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:32:20.833 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:32:20.841 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:32:20.841 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:32:20.841 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:32:20.841 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:32:20.841 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:32:20.841 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:32:20.843 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:32:20.843 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:32:20.843 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:32:20.844 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:32:20.844 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:32:20.845 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:32:20.845 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:32:21.423 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:32:21.424 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:32:21.572 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:32:22.097 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:32:22.098 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T22:32:22.772 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:32:23.424 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:32:24.615 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:32:25.325 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:32:25.325 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T22:32:26.654 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:32:26.676 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:32:26.700 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:32:26.724 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:32:26.751 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:32:26.776 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:32:26.802 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:32:26.830 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:32:26.855 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:32:26.882 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:32:26.908 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:32:26.909 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:32:26.910 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:32:26.910 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:32:26.910 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:32:26.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:32:26.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:32:26.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:32:26.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:32:26.913 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:32:26.913 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:32:26.913 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:32:26.913 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:32:26.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:32:26.964 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:32:27.577 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:32:27.577 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:32:28.274 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:32:28.847 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:32:28.847 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T22:32:29.578 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:32:29.917 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:32:31.778 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:32:32.591 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:32:32.592 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T22:32:33.024 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:32:33.044 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:32:33.076 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:32:33.097 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:32:33.123 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:32:33.157 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:32:33.181 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:32:33.216 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:32:33.249 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:32:33.270 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:32:33.297 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:32:33.298 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:32:33.298 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:32:33.298 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:32:33.298 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:32:33.299 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:32:33.299 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:32:33.299 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:32:33.299 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:32:33.299 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:32:33.299 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:32:33.300 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:32:33.300 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:32:33.300 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:32:33.976 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:32:34.026 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:32:34.027 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:32:35.126 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:32:35.649 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:32:35.650 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T22:32:36.028 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:32:36.632 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:32:38.528 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:32:39.412 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:32:39.412 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T22:32:39.714 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:32:39.745 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:32:39.761 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:32:39.788 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:32:39.816 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:32:39.840 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:32:39.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:32:39.899 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:32:39.921 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:32:39.947 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:32:39.985 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:32:40.002 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:32:40.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:32:40.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:32:40.003 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:32:40.003 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:32:40.003 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:32:40.003 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:32:40.003 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:32:40.003 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:32:40.004 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:32:40.004 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:32:40.004 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:32:40.004 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:32:40.004 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:32:41.175 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:32:41.175 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:32:42.027 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:32:42.599 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T22:32:43.042 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:32:43.192 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:32:45.793 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:32:46.553 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T22:32:46.810 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:32:46.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:32:46.889 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:32:46.921 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:32:46.958 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:32:46.974 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:32:46.988 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:32:47.033 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:32:47.078 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:32:47.106 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:32:47.136 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:32:47.172 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:32:47.172 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:32:47.172 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:32:47.172 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:32:47.172 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:32:47.173 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:32:47.173 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:32:47.173 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:32:47.173 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:32:47.173 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:32:47.173 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:32:47.173 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:32:47.174 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:32:47.174 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:32:48.167 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:32:48.168 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:32:49.040 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:32:49.040 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:32:49.040 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:32:49.041 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:32:49.041 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:32:49.614 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T22:32:49.618 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:32:50.174 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:32:52.519 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:32:52.519 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:32:52.520 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:32:52.520 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:32:53.011 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:32:53.336 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:32:53.378 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T22:32:54.215 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:32:54.252 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:32:54.284 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:32:54.313 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:32:54.341 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:32:54.387 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:32:54.427 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:32:54.463 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:32:54.494 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:32:54.520 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:32:54.522 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:32:54.522 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:32:54.522 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:32:54.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:32:54.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:32:54.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:32:54.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:32:54.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:32:54.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:32:54.523 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:32:54.524 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:32:54.524 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:32:54.524 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:32:55.354 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:32:55.354 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:32:55.801 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:32:55.842 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:32:56.363 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T22:32:57.356 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:32:58.997 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:32:59.042 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:33:00.113 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T22:33:00.609 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:33:00.647 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:33:00.681 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:33:00.708 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:33:00.736 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:33:00.763 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:33:00.789 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:33:00.815 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:33:00.854 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:33:00.888 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:33:00.914 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:33:00.917 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:33:00.917 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:33:00.917 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:33:00.917 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:33:00.918 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:33:00.918 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:33:00.918 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:33:00.918 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:33:00.918 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:33:00.918 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:33:00.918 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:33:00.919 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:33:00.919 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:33:01.862 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:33:01.863 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:33:01.902 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:33:02.453 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:33:02.453 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:33:02.453 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:33:02.454 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:33:03.100 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T22:33:03.865 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:33:05.846 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:33:05.853 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:33:05.854 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:33:05.854 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:33:05.854 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:33:05.854 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:33:06.724 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T22:33:07.019 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:33:07.871 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:33:07.922 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:33:07.959 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:33:07.997 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:33:08.032 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:33:08.059 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:33:08.101 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:33:08.143 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:33:08.187 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:33:08.219 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:33:08.225 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:33:08.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:33:08.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:33:08.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:33:08.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:33:08.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:33:08.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:33:08.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:33:08.227 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:33:08.227 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:33:08.227 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:33:08.227 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:33:08.227 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:33:08.840 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:33:08.841 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:33:08.908 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:33:08.913 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:33:09.407 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T22:33:10.842 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:33:12.032 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:33:12.063 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:33:13.265 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T22:33:13.833 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:33:13.864 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:33:13.897 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:33:13.927 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:33:13.956 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:33:13.993 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:33:14.020 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:33:14.054 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:33:14.080 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:33:14.108 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:33:14.139 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:33:14.140 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:33:14.140 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:33:14.140 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:33:14.140 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:33:14.140 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:33:14.141 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:33:14.141 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:33:14.141 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:33:14.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:33:14.142 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:33:14.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:33:14.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:33:14.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:33:14.884 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:33:14.885 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:33:15.264 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:33:15.513 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:33:15.514 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:33:15.514 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:33:15.514 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:33:15.514 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:33:16.165 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T22:33:16.886 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:33:18.998 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:33:19.040 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:33:19.040 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:33:19.040 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:33:19.041 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:33:19.041 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:33:19.924 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T22:33:20.358 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:33:21.212 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:33:21.259 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:33:21.294 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:33:21.327 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:33:21.372 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:33:21.419 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:33:21.459 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:33:21.496 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:33:21.532 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:33:21.563 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:33:21.565 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:33:21.565 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:33:21.566 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:33:21.566 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:33:21.566 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:33:21.566 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:33:21.566 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:33:21.566 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:33:21.567 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:33:21.567 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:33:21.567 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:33:21.567 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:33:21.567 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:33:21.644 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:33:22.218 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:33:22.218 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:33:22.296 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:33:23.001 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:33:23.001 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T22:33:24.219 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:33:24.508 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:33:26.292 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:33:26.882 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:33:26.883 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T22:33:27.174 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:33:27.196 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:33:27.220 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:33:27.260 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:33:27.297 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:33:27.331 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:33:27.364 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:33:27.396 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:33:27.430 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:33:27.458 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:33:27.506 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:33:27.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:33:27.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:33:27.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:33:27.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:33:27.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:33:27.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:33:27.509 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:33:27.509 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:33:27.509 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:33:27.509 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:33:27.509 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:33:27.509 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:33:27.509 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:33:27.532 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:33:28.165 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:33:28.165 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:33:29.609 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:33:30.166 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:33:30.283 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:33:30.284 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T22:33:30.860 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:33:33.023 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:33:33.253 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:33:33.282 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:33:33.310 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:33:33.334 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:33:33.360 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:33:33.387 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:33:33.412 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:33:33.438 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:33:33.464 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:33:33.497 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:33:33.517 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:33:33.520 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:33:33.520 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:33:33.520 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:33:33.520 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:33:33.520 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:33:33.522 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:33:33.522 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:33:33.522 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:33:33.522 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:33:33.522 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:33:33.522 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:33:33.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:33:33.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:33:33.605 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:33:33.606 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T22:33:33.615 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:33:34.192 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:33:34.193 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:33:36.193 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:33:36.525 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:33:37.011 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:33:37.205 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:33:37.206 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T22:33:38.728 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:33:39.582 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:33:39.623 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:33:39.658 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:33:39.693 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:33:39.715 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:33:39.743 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:33:39.752 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:33:39.769 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:33:39.796 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:33:39.841 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:33:39.842 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:33:39.869 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:33:39.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:33:39.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:33:39.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:33:39.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:33:39.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:33:39.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:33:39.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:33:39.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:33:39.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:33:39.871 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:33:39.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:33:39.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:33:39.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:33:40.546 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:33:40.547 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T22:33:40.893 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:33:40.893 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:33:42.894 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:33:43.034 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:33:43.646 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:33:43.747 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:33:43.747 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T22:33:45.801 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:33:45.833 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:33:45.857 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:33:45.882 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:33:45.910 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:33:45.937 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:33:45.968 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:33:45.996 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:33:46.021 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:33:46.048 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:33:46.079 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:33:46.079 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:33:46.080 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:33:46.080 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:33:46.080 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:33:46.080 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:33:46.080 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:33:46.080 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:33:46.081 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:33:46.081 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:33:46.081 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:33:46.081 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:33:46.081 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:33:46.318 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:33:46.663 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:33:46.664 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:33:46.754 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:33:47.535 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:33:47.535 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T22:33:48.665 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:33:48.937 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:33:50.658 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:33:51.388 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:33:51.422 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:33:51.454 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:33:51.455 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:33:51.455 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T22:33:51.480 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:33:51.506 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:33:51.546 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:33:51.562 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:33:51.608 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:33:51.665 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:33:51.704 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:33:51.706 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:33:51.725 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:33:51.728 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:33:51.729 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:33:51.729 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:33:51.729 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:33:51.729 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:33:51.730 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:33:51.730 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:33:51.730 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:33:51.730 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:33:51.730 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:33:51.731 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:33:51.731 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:33:51.731 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:33:52.501 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:33:52.502 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:33:54.288 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:33:54.503 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:33:54.723 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:33:54.934 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:33:54.934 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T22:33:57.419 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:33:57.600 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:33:57.713 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:33:58.184 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:33:58.184 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T22:33:58.271 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:33:58.337 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:33:58.369 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:33:58.402 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:33:58.431 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:33:58.465 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:33:58.487 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:33:58.513 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:33:58.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:33:58.583 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:33:58.584 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:33:58.585 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:33:58.585 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:33:58.585 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:33:58.585 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:33:58.585 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:33:58.585 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:33:58.586 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:33:58.586 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:33:58.586 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:33:58.586 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:33:58.586 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:33:58.586 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:33:59.840 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:33:59.840 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:34:00.861 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:34:01.002 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:34:01.629 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:34:01.630 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T22:34:01.841 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:34:03.782 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:34:04.026 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:34:04.517 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:34:04.517 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:34:04.518 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:34:04.535 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:34:04.562 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:34:04.598 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:34:04.616 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:34:04.645 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:34:04.668 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:34:04.695 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:34:04.731 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:34:04.733 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:34:04.734 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:34:04.735 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:34:04.735 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:34:04.735 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:34:04.735 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:34:04.735 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:34:04.736 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:34:04.737 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:34:04.737 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:34:04.737 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:34:04.737 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:34:04.737 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:34:04.768 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:34:04.769 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T22:34:05.321 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:34:05.321 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:34:07.016 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:34:07.322 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:34:07.507 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:34:08.214 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:34:08.214 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T22:34:09.629 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:34:10.024 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:34:10.030 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:34:10.056 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:34:10.081 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:34:10.105 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:34:10.143 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:34:10.213 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:34:10.244 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:34:10.279 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:34:10.312 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:34:10.341 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:34:10.346 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:34:10.346 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:34:10.346 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:34:10.347 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:34:10.347 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:34:10.347 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:34:10.347 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:34:10.348 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:34:10.348 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:34:10.348 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:34:10.348 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:34:10.349 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:34:10.349 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:34:11.225 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:34:11.225 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:34:11.587 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:34:12.129 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:34:12.137 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T22:34:13.045 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:34:13.232 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:34:15.345 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:34:15.879 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:34:15.880 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T22:34:15.991 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:34:16.018 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:34:16.044 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:34:16.067 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:34:16.107 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:34:16.130 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:34:16.146 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:34:16.158 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:34:16.172 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:34:16.201 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:34:16.226 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:34:16.252 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:34:16.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:34:16.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:34:16.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:34:16.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:34:16.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:34:16.255 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:34:16.255 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:34:16.255 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:34:16.255 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:34:16.255 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:34:16.255 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:34:16.256 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:34:16.256 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:34:17.094 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:34:17.095 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:34:18.466 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:34:19.022 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:34:19.023 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T22:34:19.095 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:34:20.244 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:34:22.025 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:34:22.614 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:34:22.614 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T22:34:22.644 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:34:22.666 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:34:22.691 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:34:22.716 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:34:22.740 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:34:22.770 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:34:22.801 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:34:22.830 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:34:22.859 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:34:22.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:34:22.912 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:34:22.913 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:34:22.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:34:22.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:34:22.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:34:22.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:34:22.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:34:22.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:34:22.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:34:22.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:34:22.915 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:34:22.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:34:22.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:34:22.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:34:23.205 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:34:23.717 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:34:23.717 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:34:25.065 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:34:25.676 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:34:25.677 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T22:34:25.720 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:34:27.029 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:34:28.489 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:34:28.527 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:34:28.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:34:28.583 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:34:28.612 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:34:28.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:34:28.668 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:34:28.695 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:34:28.719 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:34:28.721 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:34:28.749 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:34:28.786 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:34:28.788 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:34:28.789 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:34:28.789 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:34:28.789 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:34:28.789 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:34:28.789 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:34:28.790 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:34:28.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:34:28.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:34:28.791 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:34:28.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:34:28.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:34:28.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:34:29.485 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:34:29.485 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T22:34:29.663 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:34:29.663 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:34:29.740 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:34:31.669 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:34:32.111 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:34:32.849 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:34:32.850 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T22:34:33.033 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:34:35.155 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:34:35.614 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:34:36.018 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:34:36.059 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:34:36.095 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:34:36.128 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:34:36.165 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:34:36.167 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T22:34:36.168 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:34:36.189 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:34:36.224 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:34:36.271 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:34:36.299 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:34:36.326 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:34:36.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:34:36.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:34:36.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:34:36.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:34:36.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:34:36.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:34:36.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:34:36.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:34:36.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:34:36.329 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:34:36.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:34:36.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:34:36.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:34:37.297 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:34:37.297 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:34:39.126 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:34:39.298 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:34:39.893 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T22:34:40.062 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:34:42.404 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:34:42.429 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:34:42.456 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:34:42.480 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:34:42.506 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:34:42.533 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:34:42.560 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:34:42.586 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:34:42.615 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:34:42.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:34:42.670 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:34:42.670 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:34:42.671 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:34:42.671 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:34:42.671 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:34:42.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:34:42.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:34:42.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:34:42.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:34:42.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:34:42.673 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:34:42.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:34:42.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:34:42.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:34:42.703 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:34:42.703 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:34:42.703 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:34:42.704 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:34:42.704 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:34:42.766 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:34:43.393 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:34:43.393 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:34:43.704 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T22:34:45.394 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:34:46.006 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:34:46.007 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:34:46.007 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:34:46.007 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:34:46.462 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:34:46.614 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T22:34:48.198 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:34:49.052 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:34:49.081 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:34:49.111 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:34:49.140 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:34:49.166 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:34:49.180 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:34:49.193 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:34:49.221 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:34:49.246 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:34:49.272 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:34:49.299 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:34:49.301 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:34:49.301 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:34:49.301 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:34:49.301 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:34:49.301 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:34:49.301 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:34:49.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:34:49.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:34:49.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:34:49.302 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:34:49.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:34:49.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:34:49.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:34:49.502 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:34:49.914 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T22:34:50.090 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:34:50.090 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:34:52.091 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:34:52.583 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:34:53.041 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:34:53.232 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T22:34:54.796 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:34:54.820 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:34:54.845 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:34:54.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:34:54.895 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:34:54.921 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:34:54.947 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:34:54.974 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:34:55.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:34:55.029 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:34:55.055 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:34:55.057 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:34:55.057 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:34:55.057 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:34:55.057 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:34:55.058 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:34:55.058 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:34:55.058 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:34:55.058 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:34:55.058 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:34:55.058 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:34:55.058 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:34:55.058 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:34:55.059 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:34:55.556 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:34:55.556 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:34:55.832 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:34:56.179 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:34:56.180 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:34:56.180 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:34:56.180 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:34:56.919 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T22:34:57.557 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:34:59.054 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:34:59.611 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:34:59.611 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:34:59.611 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:34:59.612 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:34:59.612 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:35:00.427 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T22:35:00.801 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:35:00.831 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:35:00.857 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:35:00.882 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:35:00.913 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:35:00.944 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:35:00.993 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:35:01.021 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:35:01.060 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:35:01.103 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:35:01.145 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:35:01.152 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:35:01.153 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:35:01.153 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:35:01.153 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:35:01.153 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:35:01.153 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:35:01.153 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:35:01.154 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:35:01.154 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:35:01.154 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:35:01.154 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:35:01.154 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:35:01.154 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:35:01.871 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:35:01.871 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:35:02.101 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:35:02.732 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:35:03.332 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T22:35:03.872 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:35:04.837 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:35:06.388 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:35:06.931 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T22:35:07.051 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:35:07.075 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:35:07.098 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:35:07.122 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:35:07.150 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:35:07.186 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:35:07.204 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:35:07.230 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:35:07.256 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:35:07.283 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:35:07.309 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:35:07.311 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:35:07.312 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:35:07.312 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:35:07.312 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:35:07.312 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:35:07.312 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:35:07.312 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:35:07.312 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:35:07.313 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:35:07.313 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:35:07.313 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:35:07.313 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:35:07.313 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:35:07.646 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:35:08.100 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:35:08.100 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:35:09.593 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:35:09.593 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:35:09.593 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:35:09.593 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:35:09.594 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:35:10.101 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:35:10.243 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T22:35:11.048 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:35:13.033 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:35:13.033 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:35:13.034 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:35:13.034 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:35:13.034 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:35:13.565 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:35:13.577 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T22:35:13.593 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:35:13.617 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:35:13.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:35:13.666 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:35:13.693 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:35:13.719 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:35:13.745 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:35:13.773 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:35:13.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:35:13.831 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:35:13.832 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:35:13.832 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:35:13.833 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:35:13.833 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:35:13.833 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:35:13.833 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:35:13.833 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:35:13.833 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:35:13.834 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:35:13.834 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:35:13.834 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:35:13.834 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:35:13.834 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:35:13.844 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:35:14.500 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:35:14.500 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:35:16.241 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:35:16.501 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:35:16.887 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:35:16.888 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T22:35:18.058 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:35:19.626 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:35:19.657 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:35:19.691 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:35:19.721 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:35:19.748 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:35:19.775 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:35:19.801 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:35:19.827 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:35:19.829 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:35:19.853 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:35:19.889 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:35:19.908 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:35:19.910 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:35:19.910 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:35:19.911 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:35:19.911 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:35:19.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:35:19.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:35:19.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:35:19.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:35:19.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:35:19.912 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:35:19.913 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:35:19.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:35:19.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:35:20.648 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:35:20.648 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T22:35:20.843 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:35:20.843 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:35:21.083 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:35:22.844 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:35:23.187 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:35:24.014 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:35:24.014 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T22:35:25.094 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:35:25.255 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:35:25.273 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:35:25.292 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:35:25.316 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:35:25.354 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:35:25.373 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:35:25.405 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:35:25.434 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:35:25.449 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:35:25.476 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:35:25.505 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:35:25.505 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:35:25.505 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:35:25.505 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:35:25.505 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:35:25.506 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:35:25.506 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:35:25.506 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:35:25.506 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:35:25.506 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:35:25.506 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:35:25.506 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:35:25.507 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:35:25.507 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:35:26.411 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:35:26.411 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:35:27.009 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:35:27.573 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:35:27.574 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T22:35:27.885 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:35:28.413 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:35:30.589 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:35:31.700 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:35:31.701 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T22:35:31.746 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:35:31.769 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:35:31.796 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:35:31.826 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:35:31.857 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:35:31.884 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:35:31.911 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:35:31.937 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:35:31.965 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:35:31.991 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:35:32.017 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:35:32.020 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:35:32.020 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:35:32.020 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:35:32.020 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:35:32.021 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:35:32.021 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:35:32.021 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:35:32.021 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:35:32.021 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:35:32.021 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:35:32.021 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:35:32.021 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:35:32.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:35:32.060 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:35:32.776 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:35:32.777 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:35:34.231 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:35:34.744 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:35:34.745 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T22:35:34.778 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:35:35.160 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:35:38.046 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:35:38.165 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:35:38.190 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:35:38.242 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:35:38.281 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:35:38.316 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:35:38.349 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:35:38.379 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:35:38.517 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:35:38.517 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:35:38.517 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:35:38.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:35:38.536 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:35:38.536 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:35:38.536 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:35:38.536 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:35:38.537 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:35:38.537 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:35:38.537 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:35:38.537 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:35:38.537 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:35:38.537 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:35:38.537 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:35:38.538 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:35:38.538 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:35:38.697 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:35:38.698 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T22:35:39.116 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:35:39.132 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:35:39.213 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:35:41.020 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:35:41.133 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:35:41.611 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:35:41.611 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T22:35:41.943 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:35:44.105 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:35:44.127 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:35:44.150 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:35:44.175 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:35:44.202 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:35:44.239 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:35:44.274 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:35:44.306 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:35:44.336 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:35:44.358 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:35:44.387 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:35:44.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:35:44.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:35:44.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:35:44.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:35:44.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:35:44.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:35:44.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:35:44.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:35:44.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:35:44.390 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:35:44.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:35:44.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:35:44.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:35:44.559 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:35:44.966 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:35:44.967 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:35:45.576 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:35:45.577 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T22:35:46.967 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:35:47.070 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:35:47.800 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:35:48.505 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:35:48.506 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T22:35:49.886 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:35:49.945 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:35:49.980 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:35:50.011 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:35:50.040 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:35:50.073 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:35:50.095 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:35:50.121 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:35:50.147 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:35:50.174 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:35:50.225 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:35:50.259 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:35:50.260 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:35:50.260 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:35:50.261 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:35:50.261 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:35:50.261 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:35:50.261 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:35:50.261 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:35:50.261 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:35:50.261 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:35:50.262 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:35:50.262 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:35:50.262 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:35:50.262 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:35:51.153 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:35:51.153 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:35:51.288 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:35:52.081 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:35:52.084 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T22:35:53.095 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:35:53.153 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:35:54.975 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:35:55.645 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:35:55.646 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T22:35:55.958 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:35:55.981 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:35:56.004 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:35:56.038 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:35:56.059 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:35:56.070 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:35:56.109 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:35:56.123 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:35:56.150 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:35:56.175 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:35:56.206 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:35:56.229 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:35:56.231 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:35:56.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:35:56.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:35:56.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:35:56.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:35:56.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:35:56.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:35:56.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:35:56.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:35:56.233 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:35:56.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:35:56.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:35:56.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:35:57.053 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:35:57.053 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:35:58.411 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:35:58.999 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:35:58.999 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T22:35:59.055 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:35:59.500 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:36:01.814 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:36:01.853 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:36:01.881 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:36:01.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:36:01.937 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:36:01.965 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:36:01.990 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:36:02.018 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:36:02.020 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:36:02.044 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:36:02.071 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:36:02.100 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:36:02.100 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:36:02.100 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:36:02.100 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:36:02.100 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:36:02.100 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:36:02.101 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:36:02.101 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:36:02.101 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:36:02.101 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:36:02.101 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:36:02.101 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:36:02.102 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:36:02.102 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:36:02.378 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:36:02.736 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:36:02.738 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T22:36:03.086 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:36:03.086 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:36:05.089 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:36:05.314 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:36:05.532 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:36:05.976 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:36:05.977 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T22:36:07.697 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:36:08.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:36:08.582 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:36:08.588 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:36:08.612 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:36:08.639 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:36:08.655 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:36:08.676 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:36:08.705 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:36:08.731 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:36:08.757 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:36:08.784 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:36:08.810 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:36:08.812 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:36:08.812 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:36:08.812 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:36:08.812 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:36:08.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:36:08.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:36:08.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:36:08.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:36:08.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:36:08.813 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:36:08.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:36:08.814 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:36:08.814 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:36:09.362 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:36:09.363 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T22:36:09.681 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:36:09.682 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:36:11.683 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:36:11.885 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:36:12.076 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:36:12.627 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:36:12.628 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T22:36:14.462 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:36:14.493 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:36:14.533 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:36:14.563 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:36:14.595 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:36:14.618 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:36:14.641 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:36:14.669 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:36:14.704 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:36:14.739 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:36:14.773 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:36:14.774 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:36:14.774 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:36:14.774 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:36:14.774 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:36:14.775 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:36:14.775 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:36:14.775 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:36:14.775 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:36:14.775 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:36:14.776 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:36:14.776 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:36:14.776 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:36:14.776 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:36:15.021 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:36:15.434 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:36:15.435 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:36:15.435 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:36:16.186 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:36:16.187 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T22:36:17.444 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:36:18.094 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:36:19.294 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:36:19.959 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:36:19.960 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T22:36:20.216 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:36:20.241 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:36:20.265 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:36:20.291 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:36:20.314 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:36:20.340 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:36:20.367 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:36:20.393 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:36:20.438 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:36:20.448 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:36:20.475 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:36:20.477 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:36:20.477 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:36:20.477 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:36:20.477 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:36:20.478 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:36:20.478 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:36:20.478 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:36:20.478 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:36:20.478 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:36:20.478 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:36:20.478 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:36:20.479 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:36:20.479 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:36:20.879 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:36:21.158 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:36:21.158 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:36:22.719 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:36:23.160 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:36:23.380 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:36:23.383 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T22:36:24.130 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:36:26.057 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:36:26.087 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:36:26.105 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:36:26.128 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:36:26.154 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:36:26.182 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:36:26.208 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:36:26.234 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:36:26.268 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:36:26.285 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:36:26.324 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:36:26.363 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:36:26.363 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:36:26.363 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:36:26.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:36:26.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:36:26.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:36:26.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:36:26.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:36:26.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:36:26.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:36:26.365 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:36:26.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:36:26.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:36:26.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:36:26.991 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:36:26.991 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T22:36:27.166 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:36:27.166 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:36:27.257 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:36:29.167 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:36:29.469 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:36:30.070 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:36:30.410 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T22:36:31.920 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:36:31.944 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:36:31.983 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:36:32.020 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:36:32.049 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:36:32.078 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:36:32.104 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:36:32.131 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:36:32.182 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:36:32.237 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:36:32.293 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:36:32.296 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:36:32.296 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:36:32.296 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:36:32.297 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:36:32.297 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:36:32.297 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:36:32.298 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:36:32.299 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:36:32.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:36:32.318 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:36:32.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:36:32.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:36:32.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:36:32.981 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:36:32.981 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:36:33.073 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:36:33.301 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:36:33.301 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:36:33.301 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:36:33.301 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:36:33.302 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:36:33.981 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T22:36:34.983 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:36:36.099 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:36:37.181 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:36:37.181 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:36:37.181 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:36:37.181 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:36:37.181 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:36:37.854 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T22:36:38.126 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:36:38.169 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:36:38.177 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:36:38.198 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:36:38.225 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:36:38.251 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:36:38.277 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:36:38.316 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:36:38.330 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:36:38.357 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:36:38.385 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:36:38.386 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:36:38.386 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:36:38.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:36:38.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:36:38.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:36:38.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:36:38.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:36:38.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:36:38.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:36:38.388 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:36:38.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:36:38.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:36:38.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:36:38.878 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:36:39.039 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:36:39.039 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:36:40.306 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:36:41.041 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:36:41.122 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T22:36:42.178 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:36:43.860 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:36:43.884 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:36:43.907 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:36:43.932 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:36:43.959 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:36:43.989 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:36:44.011 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:36:44.037 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:36:44.064 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:36:44.090 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:36:44.126 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:36:44.127 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:36:44.127 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:36:44.128 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:36:44.128 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:36:44.128 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:36:44.128 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:36:44.128 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:36:44.128 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:36:44.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:36:44.129 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:36:44.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:36:44.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:36:44.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:36:44.142 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:36:44.142 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:36:44.143 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:36:44.143 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:36:44.143 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:36:44.813 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:36:44.813 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:36:45.022 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T22:36:45.137 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:36:46.814 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:36:47.813 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:36:47.813 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:36:47.814 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:36:47.814 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:36:47.814 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:36:48.109 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:36:48.480 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T22:36:50.304 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:36:50.986 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:36:50.986 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:36:50.987 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:36:50.987 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:36:51.039 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:36:51.156 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:36:51.189 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:36:51.215 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:36:51.243 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:36:51.269 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:36:51.295 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:36:51.321 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:36:51.350 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:36:51.376 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:36:51.402 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:36:51.404 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:36:51.405 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:36:51.405 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:36:51.405 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:36:51.405 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:36:51.405 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:36:51.406 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:36:51.406 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:36:51.406 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:36:51.406 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:36:51.406 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:36:51.406 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:36:51.406 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:36:51.603 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T22:36:52.287 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:36:52.287 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:36:54.253 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:36:54.288 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:36:54.385 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:36:54.954 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T22:36:57.243 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:36:57.325 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:36:57.512 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:36:58.172 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:36:58.220 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:36:58.258 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:36:58.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:36:58.325 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:36:58.355 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:36:58.382 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:36:58.409 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:36:58.436 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:36:58.463 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:36:58.465 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:36:58.465 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:36:58.465 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:36:58.465 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:36:58.466 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:36:58.466 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:36:58.466 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:36:58.466 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:36:58.466 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:36:58.466 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:36:58.466 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:36:58.467 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:36:58.467 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:36:58.516 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T22:36:59.123 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:36:59.124 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:37:01.125 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:37:01.135 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:37:01.150 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:37:01.150 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:37:01.150 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:37:01.150 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:37:02.197 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T22:37:03.791 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:37:04.174 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:37:04.993 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:37:04.993 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:37:04.993 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:37:04.993 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:37:04.994 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:37:05.030 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:37:05.059 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:37:05.079 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:37:05.107 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:37:05.132 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:37:05.159 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:37:05.185 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:37:05.229 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:37:05.270 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:37:05.317 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:37:05.317 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:37:05.317 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:37:05.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:37:05.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:37:05.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:37:05.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:37:05.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:37:05.327 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:37:05.327 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:37:05.327 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:37:05.327 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:37:05.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:37:05.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:37:05.976 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T22:37:06.287 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:37:06.288 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:37:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:37:08.289 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:37:08.492 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:37:09.236 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T22:37:09.864 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:37:11.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:37:12.180 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:37:12.223 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:37:12.255 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:37:12.263 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:37:12.300 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:37:12.326 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:37:12.369 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:37:12.401 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:37:12.431 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:37:12.461 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:37:12.487 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:37:12.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:37:12.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:37:12.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:37:12.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:37:12.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:37:12.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:37:12.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:37:12.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:37:12.491 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:37:12.491 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:37:12.491 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:37:12.491 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:37:12.491 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:37:12.879 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:37:12.879 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T22:37:13.240 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:37:13.240 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:37:14.124 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:37:15.062 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:37:15.241 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:37:15.617 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:37:15.618 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T22:37:16.734 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:37:18.210 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:37:18.253 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:37:18.274 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:37:18.304 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:37:18.332 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:37:18.359 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:37:18.385 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:37:18.412 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:37:18.442 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:37:18.481 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:37:18.517 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:37:18.520 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:37:18.520 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:37:18.521 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:37:18.521 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:37:18.521 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:37:18.521 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:37:18.524 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:37:18.524 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:37:18.524 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:37:18.524 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:37:18.524 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:37:18.525 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:37:18.525 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:37:18.825 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:37:19.336 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:37:19.337 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:37:19.554 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:37:19.554 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T22:37:21.107 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:37:21.338 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:37:21.765 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:37:22.526 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:37:22.527 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T22:37:23.892 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:37:24.169 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:37:24.169 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:37:24.169 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:37:24.187 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:37:24.214 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:37:24.240 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:37:24.266 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:37:24.294 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:37:24.320 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:37:24.359 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:37:24.385 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:37:24.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:37:24.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:37:24.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:37:24.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:37:24.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:37:24.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:37:24.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:37:24.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:37:24.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:37:24.388 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:37:24.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:37:24.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:37:24.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:37:25.391 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:37:25.391 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:37:25.871 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:37:26.532 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:37:26.532 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T22:37:27.128 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:37:27.392 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:37:29.104 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:37:29.693 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:37:29.856 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:37:29.857 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T22:37:30.260 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:37:31.121 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:37:31.172 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:37:31.211 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:37:31.249 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:37:31.281 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:37:31.324 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:37:31.351 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:37:31.384 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:37:31.418 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:37:31.450 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:37:31.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:37:31.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:37:31.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:37:31.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:37:31.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:37:31.455 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:37:31.455 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:37:31.455 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:37:31.455 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:37:31.455 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:37:31.455 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:37:31.455 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:37:31.456 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:37:32.210 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:37:32.210 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:37:32.922 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:37:32.937 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:37:33.646 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:37:33.647 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T22:37:34.211 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:37:35.866 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:37:36.166 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:37:37.041 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:37:37.042 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T22:37:37.242 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:37:37.264 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:37:37.297 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:37:37.318 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:37:37.344 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:37:37.370 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:37:37.399 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:37:37.425 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:37:37.453 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:37:37.495 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:37:37.533 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:37:37.535 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:37:37.535 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:37:37.535 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:37:37.536 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:37:37.536 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:37:37.536 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:37:37.536 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:37:37.536 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:37:37.537 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:37:37.537 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:37:37.537 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:37:37.537 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:37:37.538 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:37:38.449 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:37:38.449 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:37:39.703 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:37:40.154 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:37:40.258 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:37:40.258 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T22:37:40.451 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:37:42.729 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:37:42.949 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:37:42.973 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:37:42.997 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:37:43.022 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:37:43.052 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:37:43.079 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:37:43.106 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:37:43.153 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:37:43.159 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:37:43.186 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:37:43.209 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:37:43.213 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:37:43.213 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:37:43.214 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:37:43.214 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:37:43.214 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:37:43.214 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:37:43.214 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:37:43.214 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:37:43.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:37:43.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:37:43.215 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:37:43.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:37:43.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:37:43.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:37:44.015 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:37:44.015 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T22:37:44.338 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:37:44.339 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:37:46.087 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:37:46.339 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:37:46.707 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:37:47.487 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:37:47.487 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T22:37:48.637 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:37:49.034 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:37:49.057 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:37:49.086 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:37:49.119 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:37:49.144 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:37:49.173 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:37:49.197 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:37:49.225 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:37:49.250 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:37:49.276 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:37:49.303 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:37:49.305 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:37:49.305 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:37:49.305 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:37:49.305 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:37:49.305 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:37:49.306 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:37:49.306 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:37:49.306 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:37:49.306 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:37:49.307 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:37:49.307 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:37:49.307 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:37:49.307 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:37:50.163 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:37:50.164 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:37:50.574 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:37:51.191 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:37:51.192 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T22:37:52.131 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:37:52.165 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:37:53.983 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:37:54.657 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:37:54.658 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T22:37:55.039 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:37:55.110 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:37:55.132 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:37:55.156 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:37:55.185 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:37:55.206 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:37:55.232 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:37:55.259 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:37:55.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:37:55.336 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:37:55.374 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:37:55.408 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:37:55.413 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:37:55.413 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:37:55.414 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:37:55.414 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:37:55.414 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:37:55.414 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:37:55.414 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:37:55.414 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:37:55.414 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:37:55.414 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:37:55.415 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:37:55.415 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:37:55.415 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:37:56.443 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:37:56.443 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:37:57.601 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:37:58.174 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:37:58.175 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T22:37:58.444 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:37:58.488 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:38:01.089 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:38:01.196 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:38:01.320 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:38:01.342 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:38:01.367 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:38:01.388 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:38:01.423 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:38:01.452 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:38:01.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:38:01.510 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:38:01.538 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:38:01.565 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:38:01.591 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:38:01.592 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:38:01.593 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:38:01.593 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:38:01.593 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:38:01.593 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:38:01.594 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:38:01.594 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:38:01.594 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:38:01.594 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:38:01.595 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:38:01.595 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:38:01.595 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:38:01.595 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:38:01.766 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:38:01.766 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T22:38:02.335 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:38:02.335 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:38:04.337 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:38:04.709 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:38:05.156 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:38:05.486 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:38:05.487 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T22:38:07.201 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:38:07.222 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:38:07.244 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:38:07.268 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:38:07.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:38:07.322 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:38:07.349 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:38:07.377 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:38:07.401 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:38:07.428 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:38:07.455 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:38:07.457 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:38:07.457 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:38:07.457 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:38:07.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:38:07.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:38:07.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:38:07.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:38:07.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:38:07.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:38:07.458 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:38:07.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:38:07.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:38:07.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:38:08.047 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:38:08.071 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:38:08.071 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:38:08.464 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:38:09.036 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:38:09.036 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T22:38:10.072 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:38:10.986 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:38:12.321 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:38:13.046 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:38:13.047 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T22:38:13.480 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:38:13.522 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:38:13.523 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:38:13.544 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:38:13.570 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:38:13.597 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:38:13.623 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:38:13.651 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:38:13.676 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:38:13.706 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:38:13.738 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:38:13.739 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:38:13.739 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:38:13.739 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:38:13.740 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:38:13.740 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:38:13.740 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:38:13.740 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:38:13.740 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:38:13.740 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:38:13.740 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:38:13.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:38:13.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:38:13.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:38:13.936 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:38:14.430 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:38:14.430 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:38:15.883 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:38:16.431 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:38:16.501 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:38:16.502 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T22:38:17.151 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:38:19.251 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:38:19.560 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:38:20.104 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:38:20.106 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:38:20.122 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:38:20.122 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T22:38:20.154 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:38:20.194 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:38:20.244 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:38:20.275 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:38:20.303 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:38:20.330 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:38:20.371 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:38:20.398 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:38:20.424 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:38:20.426 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:38:20.426 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:38:20.426 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:38:20.426 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:38:20.426 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:38:20.427 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:38:20.427 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:38:20.427 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:38:20.427 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:38:20.427 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:38:20.427 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:38:20.427 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:38:20.428 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:38:21.304 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:38:21.304 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:38:23.178 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:38:23.182 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:38:23.305 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:38:23.932 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:38:23.932 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T22:38:26.282 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:38:26.450 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:38:26.474 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:38:26.498 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:38:26.523 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:38:26.550 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:38:26.576 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:38:26.608 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:38:26.630 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:38:26.656 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:38:26.687 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:38:26.713 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:38:26.714 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:38:26.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:38:26.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:38:26.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:38:26.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:38:26.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:38:26.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:38:26.716 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:38:26.716 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:38:26.716 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:38:26.716 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:38:26.716 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:38:26.716 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:38:26.750 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:38:26.750 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:38:26.751 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:38:26.751 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:38:27.580 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T22:38:27.936 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:38:27.936 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:38:29.938 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:38:30.052 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:38:30.052 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:38:30.052 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:38:30.052 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:38:30.052 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:38:30.095 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:38:30.777 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T22:38:33.090 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:38:33.241 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:38:33.266 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:38:33.288 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:38:33.312 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:38:33.357 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:38:33.366 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:38:33.392 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:38:33.423 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:38:33.449 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:38:33.475 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:38:33.477 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:38:33.501 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:38:33.503 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:38:33.503 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:38:33.503 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:38:33.504 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:38:33.504 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:38:33.504 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:38:33.504 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:38:33.504 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:38:33.505 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:38:33.505 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:38:33.505 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:38:33.505 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:38:33.505 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:38:34.479 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T22:38:34.485 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:38:34.486 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:38:36.487 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:38:36.812 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:38:37.171 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:38:37.675 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T22:38:39.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:38:39.652 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:38:39.705 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:38:39.739 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:38:39.776 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:38:39.812 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:38:39.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:38:39.881 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:38:39.911 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:38:39.942 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:38:39.972 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:38:39.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:38:39.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:38:39.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:38:39.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:38:39.976 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:38:39.976 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:38:39.976 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:38:39.976 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:38:39.977 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:38:39.977 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:38:39.977 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:38:39.977 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:38:39.978 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:38:40.298 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:38:40.466 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:38:40.466 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:38:40.466 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:38:40.467 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:38:40.467 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:38:40.602 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:38:40.602 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:38:41.181 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T22:38:42.604 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:38:43.231 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:38:44.615 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:38:44.616 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:38:44.616 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:38:44.616 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:38:44.620 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:38:45.226 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T22:38:45.493 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:38:45.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:38:45.538 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:38:45.561 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:38:45.587 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:38:45.613 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:38:45.639 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:38:45.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:38:45.693 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:38:45.722 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:38:45.748 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:38:45.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:38:45.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:38:45.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:38:45.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:38:45.751 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:38:45.751 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:38:45.751 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:38:45.751 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:38:45.751 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:38:45.751 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:38:45.752 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:38:45.752 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:38:45.752 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:38:46.106 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:38:46.326 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:38:46.326 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:38:47.650 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:38:48.327 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:38:48.355 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T22:38:50.101 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:38:51.368 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:38:51.573 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:38:52.097 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T22:38:52.221 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:38:52.244 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:38:52.269 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:38:52.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:38:52.323 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:38:52.350 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:38:52.378 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:38:52.403 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:38:52.431 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:38:52.459 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:38:52.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:38:52.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:38:52.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:38:52.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:38:52.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:38:52.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:38:52.461 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:38:52.461 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:38:52.461 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:38:52.461 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:38:52.461 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:38:52.461 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:38:52.461 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:38:52.911 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:38:53.083 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:38:53.084 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:38:55.058 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:38:55.058 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:38:55.059 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:38:55.059 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:38:55.059 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:38:55.085 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:38:55.792 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T22:38:56.096 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:38:58.269 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:38:58.305 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:38:58.329 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:38:58.354 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:38:58.381 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:38:58.428 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:38:58.474 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:38:58.513 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:38:58.553 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:38:58.554 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:38:58.580 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:38:58.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:38:58.607 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:38:58.608 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:38:58.608 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:38:58.608 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:38:58.608 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:38:58.608 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:38:58.609 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:38:58.609 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:38:58.609 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:38:58.609 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:38:58.609 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:38:58.609 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:38:58.609 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:38:59.150 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:38:59.161 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T22:38:59.366 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:38:59.366 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:39:01.367 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:39:01.783 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:39:02.304 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:39:02.667 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T22:39:04.513 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:39:05.141 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:39:05.370 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:39:05.371 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:39:05.423 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:39:05.463 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:39:05.499 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:39:05.530 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:39:05.563 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:39:05.597 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:39:05.621 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:39:05.671 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:39:05.714 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T22:39:05.721 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:39:05.723 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:39:05.723 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:39:05.723 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:39:05.723 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:39:05.723 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:39:05.723 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:39:05.724 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:39:05.724 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:39:05.724 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:39:05.724 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:39:05.725 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:39:05.725 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:39:05.725 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:39:06.386 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:39:06.387 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:39:08.388 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:39:08.429 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:39:08.585 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:39:08.978 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:39:08.979 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T22:39:11.430 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:39:11.452 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:39:11.476 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:39:11.499 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:39:11.527 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:39:11.553 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:39:11.566 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:39:11.580 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:39:11.608 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:39:11.634 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:39:11.661 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:39:11.675 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:39:11.687 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:39:11.688 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:39:11.689 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:39:11.689 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:39:11.689 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:39:11.689 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:39:11.689 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:39:11.689 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:39:11.689 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:39:11.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:39:11.690 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:39:11.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:39:11.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:39:11.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:39:12.602 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:39:12.603 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T22:39:12.771 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:39:12.771 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:39:14.711 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:39:14.773 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:39:15.233 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:39:15.935 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:39:15.936 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T22:39:17.378 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:39:17.812 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:39:18.614 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:39:18.664 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:39:18.712 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:39:18.751 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:39:18.779 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:39:18.806 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:39:18.832 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:39:18.860 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:39:18.886 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:39:18.913 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:39:18.940 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:39:18.942 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:39:18.942 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:39:18.942 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:39:18.942 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:39:18.942 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:39:18.942 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:39:18.943 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:39:18.943 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:39:18.943 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:39:18.943 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:39:18.957 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:39:18.957 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:39:18.957 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:39:19.478 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:39:19.478 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T22:39:19.641 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:39:19.641 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:39:21.177 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:39:21.642 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:39:21.700 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:39:22.229 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:39:22.229 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T22:39:23.794 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:39:24.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:39:24.484 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:39:24.514 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:39:24.545 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:39:24.585 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:39:24.624 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:39:24.661 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:39:24.687 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:39:24.715 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:39:24.742 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:39:24.768 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:39:24.769 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:39:24.771 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:39:24.771 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:39:24.771 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:39:24.771 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:39:24.771 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:39:24.772 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:39:24.772 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:39:24.772 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:39:24.772 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:39:24.772 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:39:24.772 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:39:24.772 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:39:24.773 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:39:25.520 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:39:25.520 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:39:25.893 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:39:25.893 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T22:39:27.201 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:39:27.521 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:39:28.324 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:39:29.084 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:39:29.084 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T22:39:29.910 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:39:30.024 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:39:30.069 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:39:30.102 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:39:30.142 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:39:30.170 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:39:30.206 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:39:30.232 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:39:30.255 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:39:30.278 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:39:30.314 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:39:30.332 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:39:30.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:39:30.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:39:30.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:39:30.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:39:30.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:39:30.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:39:30.335 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:39:30.335 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:39:30.335 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:39:30.335 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:39:30.335 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:39:30.335 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:39:30.335 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:39:31.427 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:39:31.427 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:39:32.340 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:39:32.897 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:39:32.898 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T22:39:33.205 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:39:33.428 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:39:35.344 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:39:35.913 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:39:35.914 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T22:39:35.992 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:39:36.518 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:39:37.371 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:39:37.424 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:39:37.480 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:39:37.513 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:39:37.546 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:39:37.574 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:39:37.601 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:39:37.627 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:39:37.655 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:39:37.682 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:39:37.683 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:39:37.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:39:37.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:39:37.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:39:37.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:39:37.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:39:37.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:39:37.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:39:37.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:39:37.685 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:39:37.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:39:37.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:39:37.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:39:38.448 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:39:38.448 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:39:38.808 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:39:39.372 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:39:39.372 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T22:39:40.117 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:39:40.450 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:39:42.373 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:39:42.814 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:39:43.025 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:39:43.058 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:39:43.096 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:39:43.096 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T22:39:43.108 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:39:43.145 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:39:43.171 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:39:43.197 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:39:43.241 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:39:43.251 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:39:43.277 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:39:43.302 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:39:43.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:39:43.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:39:43.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:39:43.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:39:43.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:39:43.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:39:43.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:39:43.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:39:43.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:39:43.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:39:43.334 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:39:43.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:39:43.335 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:39:43.335 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:39:44.113 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:39:44.113 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:39:45.742 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:39:46.113 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:39:46.196 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:39:46.290 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:39:46.290 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T22:39:48.741 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:39:48.940 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:39:49.335 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:39:49.749 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:39:49.749 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T22:39:50.193 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:39:50.237 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:39:50.277 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:39:50.314 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:39:50.347 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:39:50.376 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:39:50.403 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:39:50.430 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:39:50.457 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:39:50.484 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:39:50.486 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:39:50.486 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:39:50.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:39:50.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:39:50.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:39:50.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:39:50.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:39:50.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:39:50.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:39:50.489 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:39:50.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:39:50.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:39:50.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:39:51.214 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:39:51.215 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:39:52.493 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:39:52.498 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:39:53.206 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:39:53.206 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T22:39:53.215 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:39:55.388 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:39:55.683 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:39:56.210 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:39:56.233 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:39:56.272 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:39:56.309 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:39:56.344 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:39:56.395 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:39:56.417 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:39:56.418 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T22:39:56.445 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:39:56.468 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:39:56.496 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:39:56.524 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:39:56.553 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:39:56.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:39:56.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:39:56.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:39:56.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:39:56.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:39:56.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:39:56.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:39:56.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:39:56.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:39:56.556 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:39:56.556 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:39:56.556 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:39:56.556 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:39:57.357 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:39:57.357 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:39:58.942 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:39:59.359 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:39:59.407 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:40:00.001 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:40:00.002 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T22:40:01.499 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:40:02.120 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:40:02.141 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:40:02.170 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:40:02.193 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:40:02.225 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:40:02.251 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:40:02.275 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:40:02.302 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:40:02.328 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:40:02.356 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:40:02.382 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:40:02.385 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:40:02.385 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:40:02.385 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:40:02.386 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:40:02.386 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:40:02.386 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:40:02.386 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:40:02.386 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:40:02.386 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:40:02.386 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:40:02.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:40:02.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:40:02.986 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:40:03.634 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:40:03.635 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:40:03.950 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:40:03.950 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T22:40:05.002 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:40:05.636 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:40:06.373 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:40:06.973 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:40:06.974 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T22:40:07.918 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:40:08.360 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:40:08.389 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:40:08.432 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:40:08.470 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:40:08.506 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:40:08.538 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:40:08.567 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:40:08.620 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:40:08.669 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:40:08.704 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:40:08.740 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:40:08.743 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:40:08.743 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:40:08.743 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:40:08.743 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:40:08.743 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:40:08.744 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:40:08.744 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:40:08.744 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:40:08.744 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:40:08.744 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:40:08.744 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:40:08.744 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:40:08.745 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:40:09.688 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:40:09.688 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:40:10.468 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:40:11.077 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:40:11.077 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T22:40:11.691 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:40:12.026 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:40:14.001 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:40:14.569 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:40:14.593 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:40:14.627 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:40:14.636 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:40:14.636 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T22:40:14.652 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:40:14.679 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:40:14.706 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:40:14.732 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:40:14.763 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:40:14.793 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:40:14.821 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:40:14.848 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:40:14.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:40:14.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:40:14.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:40:14.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:40:14.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:40:14.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:40:14.851 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:40:14.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:40:14.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:40:14.852 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:40:14.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:40:14.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:40:14.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:40:15.240 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:40:15.549 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:40:15.549 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:40:17.503 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:40:17.550 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:40:18.108 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:40:18.108 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T22:40:18.308 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:40:20.737 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:40:21.040 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:40:21.041 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:40:21.041 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:40:21.041 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:40:21.176 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:40:21.591 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:40:21.598 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T22:40:21.635 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:40:21.668 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:40:21.705 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:40:21.728 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:40:21.756 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:40:21.784 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:40:21.812 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:40:21.844 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:40:21.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:40:21.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:40:21.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:40:21.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:40:21.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:40:21.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:40:21.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:40:21.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:40:21.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:40:21.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:40:21.871 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:40:21.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:40:21.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:40:21.873 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:40:22.601 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:40:22.601 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:40:24.522 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:40:24.533 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:40:24.533 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:40:24.533 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:40:24.534 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:40:24.534 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:40:24.602 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:40:25.585 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T22:40:27.210 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:40:27.865 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:40:28.201 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:40:28.722 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:40:28.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:40:28.778 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T22:40:28.813 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:40:28.850 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:40:28.881 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:40:28.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:40:28.945 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:40:28.975 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:40:29.004 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:40:29.030 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:40:29.032 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:40:29.032 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:40:29.032 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:40:29.032 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:40:29.033 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:40:29.033 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:40:29.033 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:40:29.033 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:40:29.033 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:40:29.033 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:40:29.033 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:40:29.034 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:40:29.034 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:40:30.045 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:40:30.045 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:40:30.938 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:40:31.671 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:40:32.047 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:40:32.191 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T22:40:33.575 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:40:34.428 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:40:34.987 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:40:34.987 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:40:34.988 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:40:34.988 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:40:34.988 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:40:35.289 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:40:35.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:40:35.396 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:40:35.444 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:40:35.479 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:40:35.507 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:40:35.538 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:40:35.569 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:40:35.594 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:40:35.620 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:40:35.636 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:40:35.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:40:35.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:40:35.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:40:35.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:40:35.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:40:35.638 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:40:35.638 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:40:35.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:40:35.639 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:40:35.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:40:35.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:40:35.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:40:35.805 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T22:40:36.546 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:40:36.546 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:40:37.238 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:40:38.491 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:40:38.492 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:40:38.492 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:40:38.492 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:40:38.492 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:40:38.548 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:40:39.049 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T22:40:39.919 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:40:42.120 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:40:42.152 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:40:42.185 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:40:42.221 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:40:42.256 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:40:42.256 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:40:42.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:40:42.314 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:40:42.345 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:40:42.372 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:40:42.400 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:40:42.426 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:40:42.427 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:40:42.428 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:40:42.428 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:40:42.428 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:40:42.428 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:40:42.428 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:40:42.429 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:40:42.429 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:40:42.429 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:40:42.429 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:40:42.429 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:40:42.429 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:40:42.430 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:40:42.985 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T22:40:43.253 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:40:43.454 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:40:43.455 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:40:45.460 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:40:45.549 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:40:45.994 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:40:46.099 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T22:40:48.878 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:40:49.492 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:40:49.744 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:40:49.790 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:40:49.831 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:40:49.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:40:49.899 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:40:49.915 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:40:49.930 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:40:49.962 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:40:49.984 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:40:50.013 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:40:50.038 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:40:50.045 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:40:50.045 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:40:50.045 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:40:50.045 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:40:50.045 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:40:50.045 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:40:50.046 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:40:50.046 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:40:50.046 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:40:50.046 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:40:50.046 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:40:50.046 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:40:50.046 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:40:50.138 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T22:40:50.996 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:40:50.996 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:40:52.704 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:40:52.714 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:40:52.715 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:40:52.715 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:40:52.715 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:40:52.999 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:40:53.250 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T22:40:56.134 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:40:56.148 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:40:56.168 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:40:56.191 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:40:56.227 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:40:56.260 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:40:56.293 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:40:56.319 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:40:56.345 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:40:56.372 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:40:56.399 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:40:56.425 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:40:56.430 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:40:56.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:40:56.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:40:56.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:40:56.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:40:56.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:40:56.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:40:56.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:40:56.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:40:56.433 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:40:56.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:40:56.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:40:56.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:40:56.696 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:40:57.350 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:40:57.350 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:40:57.559 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T22:40:58.834 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:40:59.352 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:40:59.723 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:41:00.704 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T22:41:02.814 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:41:03.245 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:41:03.681 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:41:03.733 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:41:03.773 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:41:03.803 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:41:03.819 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:41:03.855 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:41:03.887 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:41:03.917 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:41:03.944 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:41:03.971 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:41:03.997 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:41:04.000 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:41:04.000 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:41:04.000 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:41:04.000 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:41:04.000 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:41:04.000 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:41:04.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:41:04.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:41:04.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:41:04.001 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:41:04.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:41:04.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:41:04.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:41:04.312 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:41:04.312 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T22:41:04.703 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:41:04.704 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:41:05.806 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:41:06.355 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:41:06.705 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:41:06.941 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:41:06.942 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T22:41:09.660 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:41:10.134 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:41:10.220 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:41:10.265 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:41:10.266 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T22:41:10.266 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:41:10.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:41:10.323 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:41:10.351 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:41:10.378 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:41:10.404 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:41:10.430 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:41:10.456 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:41:10.483 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:41:10.510 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:41:10.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:41:10.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:41:10.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:41:10.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:41:10.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:41:10.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:41:10.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:41:10.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:41:10.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:41:10.513 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:41:10.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:41:10.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:41:10.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:41:11.314 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:41:11.314 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:41:12.703 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:41:12.778 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:41:13.316 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:41:13.397 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:41:13.397 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T22:41:16.140 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:41:16.433 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:41:16.451 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:41:16.478 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:41:16.501 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:41:16.528 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:41:16.554 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:41:16.580 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:41:16.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:41:16.633 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:41:16.661 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:41:16.687 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:41:16.689 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:41:16.689 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:41:16.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:41:16.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:41:16.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:41:16.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:41:16.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:41:16.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:41:16.690 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:41:16.691 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:41:16.691 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:41:16.691 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:41:16.934 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:41:17.501 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:41:17.502 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:41:17.662 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:41:17.663 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T22:41:18.990 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:41:19.503 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:41:19.845 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:41:20.715 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:41:20.716 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T22:41:23.046 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:41:23.808 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:41:24.380 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:41:24.380 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T22:41:25.756 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:41:26.454 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:41:27.079 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:41:27.080 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T22:41:28.830 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:41:29.486 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:41:30.049 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:41:30.049 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T22:41:31.523 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:41:32.080 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:41:32.712 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:41:32.713 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T22:41:32.978 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:41:33.004 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:41:33.029 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:41:33.053 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:41:33.080 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:41:33.110 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:41:33.154 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:41:33.182 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:41:33.210 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:41:33.237 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:41:33.266 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:41:33.266 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:41:33.266 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:41:33.267 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:41:33.267 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:41:33.267 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:41:33.267 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:41:33.267 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:41:33.267 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:41:33.267 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:41:33.268 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:41:33.268 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:41:33.268 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:41:33.268 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:41:34.145 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:41:34.145 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:41:35.087 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:41:35.125 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:41:35.989 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:41:35.989 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T22:41:36.146 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:41:37.783 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:41:38.515 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:41:39.322 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:41:39.322 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T22:41:40.052 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:41:40.080 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:41:40.107 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:41:40.132 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:41:40.158 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:41:40.192 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:41:40.220 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:41:40.251 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:41:40.277 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:41:40.304 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:41:40.331 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:41:40.332 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:41:40.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:41:40.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:41:40.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:41:40.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:41:40.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:41:40.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:41:40.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:41:40.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:41:40.335 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:41:40.335 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:41:40.335 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:41:40.335 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:41:41.152 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:41:41.341 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:41:41.342 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:41:42.304 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:41:42.851 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:41:42.852 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T22:41:43.343 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:41:43.919 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:41:46.116 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:41:46.335 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:41:46.770 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:41:46.771 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T22:41:47.193 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:41:47.221 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:41:47.248 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:41:47.274 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:41:47.300 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:41:47.326 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:41:47.352 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:41:47.378 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:41:47.405 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:41:47.431 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:41:47.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:41:47.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:41:47.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:41:47.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:41:47.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:41:47.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:41:47.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:41:47.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:41:47.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:41:47.434 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:41:47.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:41:47.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:41:47.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:41:48.018 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:41:48.019 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:41:48.281 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:41:49.320 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:41:49.857 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:41:49.857 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T22:41:50.020 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:41:50.914 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:41:52.774 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:41:53.257 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:41:53.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:41:53.309 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:41:53.335 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:41:53.361 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:41:53.388 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:41:53.415 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:41:53.441 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:41:53.468 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:41:53.495 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:41:53.521 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:41:53.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:41:53.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:41:53.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:41:53.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:41:53.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:41:53.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:41:53.524 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:41:53.524 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:41:53.524 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:41:53.524 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:41:53.524 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:41:53.524 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:41:53.524 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:41:53.668 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:41:53.668 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T22:41:54.386 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:41:54.386 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:41:54.697 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:41:56.387 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:41:56.400 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:41:56.981 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:41:56.981 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T22:41:57.548 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:41:59.718 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:41:59.753 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:41:59.783 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:41:59.809 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:41:59.835 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:41:59.863 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:41:59.889 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:41:59.917 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:41:59.942 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:41:59.978 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:42:00.016 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:42:00.017 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:42:00.017 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:42:00.017 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:42:00.017 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:42:00.017 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:42:00.017 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:42:00.018 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:42:00.018 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:42:00.018 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:42:00.018 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:42:00.018 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:42:00.018 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:42:00.018 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:42:00.145 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:42:00.874 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:42:00.875 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:42:01.041 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:42:01.042 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T22:42:01.150 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:42:02.876 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:42:03.328 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:42:03.693 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:42:03.976 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:42:03.976 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T22:42:06.573 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:42:06.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:42:06.634 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:42:06.661 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:42:06.690 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:42:06.717 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:42:06.748 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:42:06.773 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:42:06.798 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:42:06.829 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:42:06.854 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:42:06.855 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:42:06.856 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:42:06.856 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:42:06.856 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:42:06.856 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:42:06.856 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:42:06.856 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:42:06.856 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:42:06.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:42:06.857 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:42:06.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:42:06.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:42:06.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:42:07.052 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:42:07.283 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:42:07.634 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:42:07.634 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:42:07.972 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:42:07.972 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T22:42:09.635 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:42:10.082 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:42:10.187 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:42:11.252 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:42:11.253 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T22:42:12.572 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:42:12.615 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:42:12.645 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:42:12.671 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:42:12.707 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:42:12.740 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:42:12.764 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:42:12.779 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:42:12.803 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:42:12.837 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:42:12.862 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:42:12.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:42:12.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:42:12.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:42:12.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:42:12.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:42:12.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:42:12.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:42:12.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:42:12.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:42:12.865 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:42:12.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:42:12.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:42:12.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:42:13.825 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:42:13.825 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:42:14.291 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:42:14.443 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:42:15.212 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T22:42:15.826 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:42:17.057 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:42:17.892 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:42:18.773 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T22:42:19.205 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:42:19.235 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:42:19.261 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:42:19.283 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:42:19.330 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:42:19.363 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:42:19.397 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:42:19.426 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:42:19.458 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:42:19.482 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:42:19.508 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:42:19.510 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:42:19.510 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:42:19.510 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:42:19.510 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:42:19.510 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:42:19.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:42:19.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:42:19.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:42:19.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:42:19.511 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:42:19.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:42:19.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:42:19.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:42:20.461 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:42:20.462 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:42:21.159 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:42:21.423 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:42:21.424 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:42:21.424 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:42:21.424 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:42:21.424 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:42:22.211 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T22:42:22.463 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:42:24.027 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:42:24.767 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:42:24.768 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:42:24.768 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:42:24.768 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:42:25.136 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:42:25.146 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:42:25.170 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:42:25.193 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:42:25.220 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:42:25.247 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:42:25.276 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:42:25.311 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:42:25.329 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:42:25.356 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:42:25.383 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:42:25.385 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:42:25.386 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:42:25.386 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:42:25.386 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:42:25.386 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:42:25.386 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:42:25.386 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:42:25.386 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:42:25.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:42:25.387 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:42:25.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:42:25.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:42:25.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:42:25.592 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T22:42:26.454 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:42:26.455 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:42:28.228 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:42:28.303 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:42:28.456 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:42:28.991 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T22:42:31.250 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:42:31.367 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:42:31.393 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:42:31.428 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:42:31.455 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:42:31.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:42:31.510 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:42:31.537 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:42:31.564 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:42:31.590 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:42:31.620 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:42:31.644 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:42:31.645 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:42:31.645 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:42:31.645 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:42:31.646 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:42:31.646 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:42:31.646 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:42:31.646 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:42:31.646 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:42:31.646 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:42:31.647 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:42:31.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:42:31.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:42:31.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:42:31.676 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:42:31.676 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:42:31.676 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:42:31.676 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:42:31.677 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:42:32.625 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:42:32.625 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:42:32.962 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T22:42:34.301 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:42:34.626 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:42:35.722 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:42:35.722 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:42:35.722 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:42:35.722 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:42:36.327 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T22:42:37.248 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:42:37.546 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:42:37.568 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:42:37.590 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:42:37.615 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:42:37.642 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:42:37.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:42:37.693 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:42:37.719 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:42:37.753 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:42:37.787 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:42:37.822 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:42:37.824 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:42:37.825 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:42:37.825 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:42:37.825 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:42:37.825 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:42:37.825 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:42:37.825 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:42:37.826 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:42:37.826 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:42:37.826 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:42:37.826 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:42:37.826 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:42:37.826 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:42:38.675 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:42:38.675 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:42:39.646 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:42:40.226 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T22:42:40.419 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:42:40.676 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:42:43.214 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:42:43.219 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:42:43.303 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:42:43.771 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T22:42:44.156 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:42:44.188 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:42:44.223 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:42:44.258 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:42:44.288 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:42:44.317 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:42:44.347 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:42:44.374 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:42:44.403 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:42:44.433 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:42:44.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:42:44.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:42:44.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:42:44.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:42:44.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:42:44.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:42:44.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:42:44.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:42:44.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:42:44.435 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:42:44.435 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:42:44.435 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:42:44.439 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:42:45.593 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:42:45.594 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:42:46.550 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:42:46.550 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:42:46.550 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:42:46.550 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:42:47.074 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T22:42:47.311 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:42:47.595 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:42:49.916 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:42:49.916 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:42:49.916 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:42:49.916 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:42:49.916 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:42:50.115 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:42:50.503 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:42:50.509 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T22:42:50.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:42:50.561 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:42:50.591 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:42:50.620 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:42:50.645 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:42:50.671 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:42:50.697 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:42:50.723 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:42:50.750 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:42:50.776 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:42:50.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:42:50.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:42:50.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:42:50.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:42:50.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:42:50.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:42:50.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:42:50.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:42:50.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:42:50.778 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:42:50.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:42:50.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:42:50.779 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:42:51.551 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:42:51.551 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:42:53.302 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:42:53.552 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:42:53.661 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:42:53.893 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T22:42:56.442 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:42:56.455 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:42:56.466 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:42:56.477 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:42:56.482 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:42:56.509 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:42:56.536 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:42:56.562 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:42:56.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:42:56.655 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:42:56.696 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:42:56.724 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:42:56.750 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:42:56.751 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:42:56.751 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:42:56.752 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:42:56.752 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:42:56.752 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:42:56.752 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:42:56.752 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:42:56.752 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:42:56.752 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:42:56.753 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:42:56.753 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:42:56.753 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:42:56.753 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:42:57.270 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:42:57.270 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T22:42:57.692 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:42:57.692 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:42:59.693 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:42:59.752 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:43:00.163 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:43:00.454 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:43:00.454 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T22:43:02.746 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:43:02.932 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:43:02.971 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:43:03.555 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:43:03.555 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T22:43:03.609 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:43:03.659 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:43:03.691 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:43:03.721 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:43:03.749 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:43:03.781 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:43:03.802 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:43:03.832 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:43:03.861 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:43:03.889 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:43:03.889 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:43:03.889 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:43:03.889 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:43:03.889 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:43:03.890 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:43:03.890 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:43:03.890 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:43:03.890 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:43:03.890 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:43:03.890 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:43:03.890 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:43:03.891 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:43:03.891 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:43:05.064 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:43:05.064 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:43:06.268 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:43:06.359 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:43:06.994 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:43:06.994 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T22:43:07.065 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:43:09.231 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:43:09.378 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:43:09.875 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:43:09.910 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:43:09.919 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:43:09.919 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T22:43:09.936 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:43:09.960 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:43:09.986 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:43:10.013 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:43:10.041 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:43:10.072 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:43:10.094 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:43:10.122 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:43:10.159 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:43:10.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:43:10.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:43:10.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:43:10.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:43:10.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:43:10.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:43:10.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:43:10.162 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:43:10.162 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:43:10.163 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:43:10.163 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:43:10.163 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:43:10.163 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:43:10.944 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:43:10.945 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:43:12.691 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:43:12.945 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:43:13.241 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:43:13.242 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T22:43:13.331 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:43:15.827 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:43:15.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:43:15.883 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:43:15.900 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:43:15.931 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:43:15.955 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:43:15.985 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:43:16.005 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:43:16.031 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:43:16.058 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:43:16.064 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:43:16.073 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:43:16.085 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:43:16.092 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:43:16.092 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:43:16.093 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:43:16.093 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:43:16.093 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:43:16.093 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:43:16.093 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:43:16.093 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:43:16.094 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:43:16.094 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:43:16.094 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:43:16.094 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:43:16.094 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:43:16.778 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:43:16.779 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T22:43:17.089 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:43:17.090 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:43:19.100 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:43:19.389 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:43:20.153 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:43:20.153 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T22:43:20.169 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:43:21.782 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:43:21.824 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:43:21.859 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:43:21.889 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:43:21.919 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:43:21.947 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:43:22.007 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:43:22.051 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:43:22.092 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:43:22.129 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:43:22.163 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:43:22.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:43:22.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:43:22.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:43:22.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:43:22.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:43:22.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:43:22.167 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:43:22.167 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:43:22.167 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:43:22.167 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:43:22.167 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:43:22.167 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:43:22.167 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:43:22.888 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:43:22.889 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:43:23.042 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:43:23.234 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:43:23.953 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:43:23.953 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T22:43:24.890 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:43:26.365 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:43:26.789 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:43:27.875 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:43:27.875 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T22:43:28.251 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:43:28.274 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:43:28.297 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:43:28.321 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:43:28.347 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:43:28.374 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:43:28.399 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:43:28.426 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:43:28.453 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:43:28.481 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:43:28.508 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:43:28.509 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:43:28.510 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:43:28.510 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:43:28.510 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:43:28.510 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:43:28.510 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:43:28.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:43:28.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:43:28.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:43:28.511 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:43:28.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:43:28.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:43:28.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:43:29.145 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:43:29.145 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:43:29.468 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:43:30.226 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:43:30.863 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:43:30.863 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T22:43:31.147 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:43:33.332 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:43:33.847 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:43:34.644 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:43:34.644 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T22:43:34.747 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:43:34.770 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:43:34.794 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:43:34.819 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:43:34.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:43:34.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:43:34.917 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:43:34.946 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:43:34.974 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:43:35.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:43:35.029 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:43:35.030 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:43:35.031 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:43:35.031 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:43:35.031 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:43:35.031 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:43:35.031 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:43:35.031 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:43:35.031 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:43:35.032 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:43:35.032 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:43:35.032 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:43:35.032 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:43:35.032 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:43:35.741 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:43:35.741 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:43:36.191 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:43:36.979 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:43:37.743 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:43:37.917 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:43:37.917 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T22:43:38.859 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:43:40.882 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:43:41.221 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:43:41.260 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:43:41.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:43:41.332 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:43:41.362 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:43:41.389 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:43:41.416 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:43:41.442 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:43:41.469 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:43:41.496 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:43:41.525 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:43:41.528 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:43:41.528 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:43:41.528 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:43:41.528 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:43:41.529 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:43:41.529 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:43:41.529 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:43:41.529 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:43:41.529 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:43:41.529 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:43:41.529 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:43:41.530 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:43:41.530 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:43:41.543 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:43:41.544 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T22:43:41.583 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:43:42.024 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:43:42.024 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:43:44.029 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:43:44.467 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:43:45.065 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:43:45.489 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:43:45.552 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T22:43:47.367 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:43:47.394 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:43:47.429 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:43:47.448 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:43:47.474 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:43:47.501 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:43:47.553 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:43:47.598 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:43:47.630 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:43:47.660 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:43:47.690 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:43:47.692 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:43:47.692 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:43:47.692 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:43:47.698 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:43:47.699 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:43:47.699 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:43:47.699 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:43:47.699 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:43:47.699 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:43:47.699 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:43:47.699 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:43:47.700 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:43:47.700 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:43:47.865 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:43:47.917 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:43:48.518 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:43:48.518 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:43:48.968 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:43:48.969 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T22:43:50.520 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:43:52.037 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:43:52.040 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:43:52.955 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:43:52.955 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T22:43:53.339 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:43:53.361 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:43:53.384 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:43:53.412 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:43:53.434 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:43:53.468 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:43:53.489 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:43:53.512 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:43:53.538 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:43:53.565 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:43:53.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:43:53.608 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:43:53.608 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:43:53.609 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:43:53.609 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:43:53.609 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:43:53.609 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:43:53.609 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:43:53.609 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:43:53.609 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:43:53.610 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:43:53.610 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:43:53.610 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:43:53.610 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:43:54.299 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:43:54.300 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:43:55.154 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:43:55.428 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:43:56.142 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:43:56.143 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T22:43:56.301 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:43:58.886 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:43:59.298 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:44:00.168 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:44:00.187 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:44:00.187 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T22:44:00.213 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:44:00.247 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:44:00.278 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:44:00.310 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:44:00.364 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:44:00.413 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:44:00.451 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:44:00.478 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:44:00.508 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:44:00.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:44:00.535 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:44:00.536 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:44:00.536 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:44:00.536 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:44:00.536 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:44:00.537 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:44:00.537 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:44:00.537 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:44:00.537 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:44:00.537 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:44:00.538 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:44:00.538 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:44:00.538 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:44:01.200 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:44:01.201 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:44:01.688 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:44:03.003 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:44:03.202 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:44:03.888 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:44:03.888 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T22:44:05.404 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:44:05.953 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:44:05.987 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:44:06.026 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:44:06.051 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:44:06.080 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:44:06.111 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:44:06.136 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:44:06.164 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:44:06.194 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:44:06.221 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:44:06.247 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:44:06.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:44:06.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:44:06.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:44:06.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:44:06.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:44:06.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:44:06.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:44:06.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:44:06.255 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:44:06.255 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:44:06.255 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:44:06.255 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:44:06.255 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:44:06.972 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:44:06.973 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:44:07.509 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:44:08.084 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:44:08.085 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T22:44:08.279 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:44:08.976 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:44:11.220 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:44:11.969 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T22:44:12.714 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:44:13.045 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:44:13.578 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:44:13.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:44:13.652 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:44:13.690 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:44:13.728 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:44:13.760 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:44:13.791 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:44:13.821 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:44:13.860 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:44:13.880 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:44:13.883 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:44:13.883 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:44:13.883 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:44:13.883 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:44:13.883 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:44:13.883 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:44:13.884 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:44:13.884 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:44:13.884 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:44:13.884 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:44:13.884 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:44:13.884 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:44:13.884 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:44:14.584 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:44:14.584 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:44:14.584 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:44:14.584 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:44:14.585 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:44:14.658 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:44:14.658 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:44:15.123 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T22:44:15.876 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:44:16.661 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:44:17.510 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:44:17.510 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:44:17.511 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:44:17.511 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:44:17.511 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:44:18.418 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T22:44:20.096 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:44:20.134 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:44:20.166 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:44:20.179 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:44:20.194 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:44:20.241 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:44:20.278 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:44:20.315 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:44:20.352 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:44:20.377 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:44:20.409 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:44:20.440 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:44:20.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:44:20.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:44:20.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:44:20.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:44:20.443 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:44:20.443 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:44:20.443 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:44:20.443 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:44:20.443 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:44:20.443 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:44:20.444 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:44:20.444 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:44:20.444 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:44:21.303 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:44:21.304 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:44:21.881 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:44:22.520 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T22:44:22.978 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:44:23.305 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:44:25.716 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:44:26.480 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T22:44:26.948 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:44:26.985 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:44:27.015 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:44:27.044 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:44:27.073 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:44:27.103 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:44:27.131 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:44:27.157 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:44:27.183 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:44:27.210 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:44:27.236 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:44:27.238 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:44:27.238 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:44:27.238 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:44:27.238 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:44:27.238 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:44:27.238 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:44:27.239 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:44:27.239 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:44:27.239 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:44:27.239 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:44:27.239 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:44:27.239 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:44:27.240 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:44:27.386 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:44:28.305 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:44:28.305 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:44:29.150 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:44:29.150 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:44:29.150 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:44:29.151 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:44:29.151 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:44:29.703 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T22:44:30.307 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:44:30.586 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:44:32.921 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:44:32.921 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:44:32.921 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:44:32.921 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:44:33.745 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T22:44:34.078 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:44:34.101 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:44:34.123 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:44:34.147 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:44:34.172 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:44:34.205 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:44:34.229 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:44:34.255 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:44:34.285 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:44:34.314 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:44:34.341 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:44:34.343 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:44:34.343 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:44:34.343 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:44:34.343 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:44:34.344 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:44:34.344 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:44:34.344 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:44:34.344 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:44:34.345 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:44:34.345 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:44:34.345 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:44:34.345 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:44:34.345 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:44:34.373 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:44:35.553 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:44:35.553 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:44:36.410 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:44:36.960 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T22:44:37.229 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:44:37.559 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:44:40.090 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:44:40.865 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T22:44:40.930 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:44:40.981 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:44:41.006 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:44:41.030 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:44:41.053 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:44:41.085 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:44:41.108 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:44:41.134 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:44:41.161 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:44:41.187 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:44:41.229 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:44:41.278 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:44:41.280 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:44:41.280 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:44:41.280 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:44:41.281 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:44:41.281 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:44:41.281 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:44:41.281 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:44:41.281 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:44:41.281 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:44:41.281 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:44:41.281 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:44:41.282 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:44:41.282 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:44:42.073 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:44:42.073 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:44:43.508 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:44:43.628 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:44:43.628 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:44:43.628 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:44:43.629 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:44:44.075 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:44:44.377 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T22:44:47.342 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:44:47.366 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:44:47.391 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:44:47.425 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:44:47.451 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:44:47.482 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:44:47.509 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:44:47.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:44:47.562 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:44:47.588 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:44:47.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:44:47.615 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:44:47.615 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:44:47.615 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:44:47.616 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:44:47.616 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:44:47.616 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:44:47.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:44:47.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:44:47.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:44:47.617 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:44:47.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:44:47.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:44:47.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:44:47.630 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:44:47.983 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:44:48.358 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:44:48.358 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:44:48.693 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T22:44:50.359 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:44:50.805 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:44:50.899 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:44:51.548 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T22:44:53.799 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:44:53.831 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:44:53.857 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:44:53.881 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:44:53.907 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:44:53.936 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:44:53.967 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:44:53.989 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:44:54.016 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:44:54.056 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:44:54.075 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:44:54.078 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:44:54.078 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:44:54.078 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:44:54.078 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:44:54.080 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:44:54.080 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:44:54.080 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:44:54.080 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:44:54.081 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:44:54.081 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:44:54.081 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:44:54.081 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:44:54.081 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:44:54.805 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:44:54.928 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:44:54.928 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:44:55.339 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:44:55.339 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T22:44:55.388 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:44:56.931 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:44:57.687 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:44:57.924 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:44:58.339 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:44:58.339 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T22:45:00.100 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:45:00.977 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:45:01.002 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:45:01.036 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:45:01.073 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:45:01.095 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:45:01.120 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:45:01.147 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:45:01.173 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:45:01.202 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:45:01.229 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:45:01.231 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:45:01.231 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:45:01.231 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:45:01.231 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:45:01.231 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:45:01.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:45:01.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:45:01.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:45:01.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:45:01.232 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:45:01.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:45:01.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:45:01.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:45:01.493 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:45:01.902 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:45:01.902 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:45:02.197 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:45:02.348 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:45:02.349 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T22:45:03.904 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:45:04.483 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:45:04.782 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:45:05.092 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:45:05.092 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T22:45:07.308 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:45:07.332 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:45:07.367 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:45:07.398 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:45:07.506 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:45:07.507 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:45:07.507 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:45:07.513 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:45:07.544 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:45:07.578 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:45:07.609 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:45:07.611 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:45:07.611 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:45:07.611 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:45:07.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:45:07.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:45:07.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:45:07.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:45:07.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:45:07.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:45:07.613 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:45:07.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:45:07.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:45:07.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:45:08.271 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:45:08.395 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:45:08.416 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:45:08.416 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:45:09.004 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:45:09.004 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T22:45:10.417 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:45:11.163 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:45:11.231 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:45:12.465 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:45:12.466 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T22:45:13.327 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:45:13.350 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:45:13.375 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:45:13.401 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:45:13.426 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:45:13.452 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:45:13.478 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:45:13.504 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:45:13.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:45:13.579 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:45:13.620 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:45:13.622 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:45:13.623 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:45:13.623 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:45:13.623 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:45:13.623 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:45:13.623 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:45:13.623 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:45:13.624 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:45:13.624 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:45:13.624 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:45:13.624 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:45:13.624 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:45:13.624 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:45:14.498 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:45:14.498 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:45:15.403 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:45:15.534 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:45:16.344 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:45:16.345 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T22:45:16.500 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:45:18.147 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:45:18.995 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:45:19.931 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:45:19.931 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:45:19.931 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:45:19.932 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:45:19.961 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:45:19.985 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:45:20.011 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:45:20.037 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:45:20.059 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:45:20.060 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T22:45:20.071 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:45:20.118 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:45:20.155 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:45:20.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:45:20.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:45:20.158 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:45:20.158 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:45:20.158 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:45:20.158 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:45:20.158 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:45:20.158 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:45:20.158 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:45:20.159 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:45:20.159 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:45:20.159 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:45:20.159 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:45:20.741 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:45:20.741 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:45:22.398 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:45:22.743 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:45:22.903 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:45:23.700 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:45:23.701 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T22:45:25.067 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:45:25.562 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:45:25.585 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:45:25.610 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:45:25.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:45:25.669 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:45:25.695 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:45:25.723 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:45:25.760 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:45:25.796 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:45:25.827 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:45:25.854 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:45:25.856 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:45:25.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:45:25.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:45:25.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:45:25.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:45:25.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:45:25.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:45:25.858 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:45:25.858 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:45:25.859 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:45:25.859 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:45:25.859 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:45:25.860 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:45:26.682 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:45:26.682 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:45:27.171 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:45:27.873 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:45:27.874 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T22:45:28.460 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:45:28.683 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:45:30.740 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:45:31.083 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:45:31.392 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:45:31.392 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T22:45:31.691 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:45:32.545 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:45:32.594 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:45:32.650 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:45:32.687 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:45:32.722 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:45:32.755 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:45:32.784 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:45:32.810 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:45:32.836 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:45:32.862 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:45:32.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:45:32.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:45:32.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:45:32.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:45:32.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:45:32.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:45:32.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:45:32.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:45:32.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:45:32.866 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:45:32.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:45:32.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:45:32.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:45:33.722 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:45:33.723 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:45:34.405 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:45:34.405 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:45:35.208 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:45:35.208 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T22:45:35.724 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:45:37.379 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:45:37.611 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:45:38.368 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:45:38.368 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T22:45:39.538 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:45:39.539 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:45:39.539 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:45:39.539 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:45:39.539 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:45:39.539 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:45:39.540 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:45:39.540 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:45:39.540 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:45:39.540 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:45:39.540 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:45:39.540 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:45:39.541 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:45:39.541 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:45:39.541 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:45:39.541 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:45:39.541 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:45:39.542 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:45:39.542 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:45:39.542 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:45:39.542 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:45:39.542 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:45:39.542 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:45:39.543 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:45:40.129 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:45:40.129 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:45:41.511 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:45:42.045 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:45:42.107 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:45:42.107 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T22:45:42.130 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:45:44.748 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:45:44.882 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:45:44.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:45:44.941 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:45:44.958 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:45:44.984 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:45:45.012 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:45:45.038 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:45:45.064 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:45:45.089 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:45:45.092 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:45:45.115 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:45:45.141 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:45:45.143 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:45:45.143 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:45:45.143 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:45:45.143 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:45:45.143 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:45:45.143 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:45:45.143 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:45:45.144 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:45:45.144 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:45:45.144 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:45:45.144 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:45:45.144 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:45:45.144 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:45:45.978 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:45:45.979 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T22:45:46.228 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:45:46.228 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:45:48.233 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:45:48.414 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:45:48.552 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:45:49.562 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:45:49.563 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T22:45:51.232 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:45:51.297 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:45:52.159 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:45:52.184 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:45:52.209 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:45:52.235 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:45:52.282 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:45:52.327 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:45:52.353 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:45:52.370 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:45:52.381 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:45:52.407 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:45:52.441 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:45:52.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:45:52.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:45:52.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:45:52.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:45:52.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:45:52.443 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:45:52.443 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:45:52.443 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:45:52.443 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:45:52.443 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:45:52.443 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:45:52.444 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:45:52.444 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:45:53.619 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:45:53.619 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:45:53.619 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:45:53.620 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T22:45:55.064 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:45:55.620 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:45:55.994 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:45:56.643 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:45:56.643 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T22:45:57.801 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:45:58.313 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:45:59.165 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:45:59.210 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:45:59.246 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:45:59.280 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:45:59.308 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:45:59.335 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:45:59.370 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:45:59.389 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:45:59.421 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:45:59.448 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:45:59.448 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:45:59.448 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:45:59.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:45:59.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:45:59.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:45:59.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:45:59.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:45:59.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:45:59.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:45:59.449 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:45:59.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:45:59.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:45:59.451 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:45:59.639 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:46:00.402 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:46:00.402 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:46:00.688 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:46:00.689 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T22:46:02.047 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:46:02.403 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:46:03.106 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:46:03.725 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:46:03.725 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T22:46:04.775 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:46:05.406 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:46:05.430 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:46:05.454 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:46:05.497 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:46:05.533 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:46:05.564 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:46:05.612 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:46:05.660 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:46:05.696 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:46:05.733 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:46:05.763 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:46:05.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:46:05.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:46:05.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:46:05.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:46:05.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:46:05.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:46:05.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:46:05.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:46:05.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:46:05.766 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:46:05.767 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:46:05.767 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:46:05.767 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:46:06.616 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:46:07.210 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:46:07.210 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:46:07.445 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:46:07.445 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T22:46:08.423 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:46:09.212 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:46:10.032 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:46:10.662 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T22:46:11.320 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:46:11.908 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:46:11.947 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:46:11.998 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:46:12.011 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:46:12.046 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:46:12.079 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:46:12.111 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:46:12.142 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:46:12.171 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:46:12.200 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:46:12.231 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:46:12.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:46:12.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:46:12.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:46:12.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:46:12.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:46:12.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:46:12.234 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:46:12.234 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:46:12.234 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:46:12.234 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:46:12.234 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:46:12.234 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:46:12.234 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:46:12.969 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:46:12.969 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:46:13.880 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:46:13.880 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:46:13.880 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:46:13.880 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:46:13.881 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:46:14.543 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T22:46:14.970 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:46:17.205 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:46:17.206 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:46:17.206 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:46:17.206 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:46:17.206 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:46:17.699 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:46:17.732 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:46:17.777 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:46:17.782 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T22:46:17.830 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:46:17.869 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:46:17.904 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:46:17.930 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:46:17.957 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:46:17.983 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:46:18.010 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:46:18.037 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:46:18.039 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:46:18.039 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:46:18.039 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:46:18.040 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:46:18.040 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:46:18.040 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:46:18.040 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:46:18.040 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:46:18.040 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:46:18.040 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:46:18.041 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:46:18.041 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:46:18.041 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:46:18.755 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:46:18.755 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:46:20.376 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:46:20.757 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:46:20.955 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T22:46:23.450 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:46:23.473 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:46:23.498 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:46:23.521 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:46:23.548 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:46:23.574 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:46:23.601 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:46:23.627 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:46:23.654 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:46:23.681 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:46:23.706 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:46:23.708 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:46:23.708 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:46:23.708 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:46:23.708 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:46:23.708 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:46:23.708 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:46:23.709 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:46:23.709 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:46:23.709 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:46:23.709 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:46:23.709 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:46:23.709 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:46:23.709 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:46:23.725 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:46:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:46:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:46:23.725 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:46:23.725 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:46:24.463 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:46:24.464 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:46:24.666 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T22:46:25.087 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:46:26.472 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:46:26.781 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:46:26.781 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:46:26.782 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:46:26.782 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:46:26.782 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:46:27.581 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T22:46:27.726 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:46:29.664 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:46:30.519 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:46:30.553 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:46:30.583 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:46:30.618 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:46:30.642 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:46:30.670 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:46:30.696 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:46:30.723 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:46:30.755 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:46:30.768 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:46:30.775 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:46:30.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:46:30.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:46:30.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:46:30.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:46:30.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:46:30.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:46:30.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:46:30.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:46:30.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:46:30.778 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:46:30.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:46:30.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:46:30.779 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:46:31.427 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:46:31.513 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T22:46:31.735 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:46:31.735 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:46:33.736 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:46:34.013 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:46:34.174 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:46:34.757 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T22:46:37.158 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:46:37.955 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:46:38.014 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:46:38.065 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:46:38.105 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:46:38.147 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:46:38.180 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:46:38.208 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:46:38.235 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:46:38.264 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:46:38.291 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:46:38.318 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:46:38.320 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:46:38.320 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:46:38.320 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:46:38.320 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:46:38.321 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:46:38.321 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:46:38.321 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:46:38.321 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:46:38.321 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:46:38.321 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:46:38.322 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:46:38.322 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:46:38.322 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:46:38.445 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:46:38.530 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T22:46:39.173 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:46:39.173 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:46:41.174 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:46:41.235 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:46:41.235 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:46:41.239 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:46:41.239 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:46:41.535 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:46:41.854 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T22:46:44.699 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:46:44.727 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:46:44.761 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:46:44.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:46:44.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:46:44.850 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:46:44.889 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:46:44.916 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:46:44.956 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:46:44.983 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:46:45.012 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:46:45.013 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:46:45.013 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:46:45.013 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:46:45.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:46:45.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:46:45.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:46:45.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:46:45.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:46:45.015 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:46:45.015 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:46:45.015 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:46:45.015 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:46:45.015 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:46:45.083 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:46:45.630 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:46:45.752 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:46:45.752 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:46:45.966 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T22:46:47.753 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:46:48.181 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:46:48.418 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:46:48.880 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T22:46:51.197 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:46:51.221 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:46:51.244 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:46:51.269 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:46:51.313 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:46:51.349 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:46:51.383 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:46:51.420 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:46:51.451 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:46:51.496 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:46:51.531 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:46:51.532 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:46:51.533 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:46:51.533 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:46:51.533 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:46:51.533 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:46:51.533 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:46:51.533 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:46:51.533 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:46:51.534 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:46:51.534 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:46:51.534 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:46:51.534 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:46:51.534 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:46:52.245 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:46:52.246 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:46:52.321 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:46:52.433 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:46:53.022 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:46:53.023 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T22:46:54.247 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:46:55.012 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:46:55.203 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:46:55.616 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:46:55.616 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T22:46:57.553 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:46:57.588 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:46:57.639 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:46:57.677 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:46:57.713 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:46:57.745 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:46:57.773 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:46:57.801 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:46:57.828 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:46:57.856 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:46:57.884 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:46:57.885 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:46:57.885 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:46:57.885 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:46:57.885 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:46:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:46:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:46:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:46:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:46:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:46:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:46:57.886 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:46:57.887 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:46:57.887 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:46:58.822 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:46:58.822 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:46:59.135 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:46:59.458 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:46:59.766 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:46:59.767 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T22:47:00.823 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:47:02.177 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:47:02.505 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:47:03.415 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:47:03.415 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T22:47:04.022 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:47:04.885 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:47:04.915 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:47:04.943 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:47:04.972 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:47:05.000 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:47:05.029 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:47:05.056 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:47:05.089 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:47:05.110 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:47:05.138 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:47:05.147 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:47:05.148 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:47:05.148 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:47:05.148 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:47:05.148 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:47:05.148 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:47:05.148 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:47:05.149 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:47:05.149 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:47:05.149 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:47:05.149 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:47:05.149 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:47:05.149 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:47:05.595 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:47:05.994 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:47:05.994 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:47:06.723 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:47:07.289 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:47:07.290 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T22:47:07.996 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:47:08.441 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:47:10.515 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:47:11.235 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:47:11.475 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:47:11.475 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T22:47:12.090 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:47:12.124 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:47:12.156 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:47:12.185 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:47:12.213 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:47:12.241 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:47:12.266 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:47:12.292 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:47:12.342 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:47:12.371 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:47:12.372 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:47:12.372 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:47:12.372 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:47:12.372 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:47:12.372 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:47:12.372 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:47:12.373 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:47:12.373 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:47:12.373 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:47:12.373 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:47:12.373 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:47:12.373 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:47:12.373 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:47:12.443 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:47:13.639 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:47:13.640 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:47:14.240 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:47:14.847 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:47:14.848 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T22:47:15.641 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:47:15.643 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:47:17.947 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:47:18.814 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:47:18.815 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T22:47:19.518 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:47:19.545 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:47:19.565 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:47:19.587 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:47:19.618 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:47:19.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:47:19.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:47:19.693 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:47:19.735 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:47:19.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:47:19.816 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:47:19.817 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:47:19.817 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:47:19.818 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:47:19.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:47:19.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:47:19.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:47:19.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:47:19.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:47:19.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:47:19.821 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:47:19.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:47:19.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:47:19.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:47:20.216 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:47:20.887 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:47:20.887 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:47:21.663 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:47:22.279 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:47:22.279 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T22:47:22.892 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:47:23.212 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:47:25.292 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:47:26.130 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:47:26.130 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T22:47:26.287 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:47:26.310 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:47:26.334 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:47:26.360 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:47:26.389 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:47:26.414 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:47:26.443 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:47:26.473 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:47:26.503 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:47:26.529 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:47:26.555 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:47:26.557 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:47:26.557 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:47:26.557 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:47:26.557 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:47:26.557 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:47:26.557 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:47:26.558 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:47:26.558 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:47:26.558 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:47:26.560 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:47:26.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:47:26.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:47:26.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:47:27.284 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:47:27.284 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:47:27.481 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:47:28.618 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:47:29.184 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:47:29.184 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T22:47:29.285 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:47:30.052 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:47:32.068 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:47:32.092 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:47:32.116 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:47:32.139 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:47:32.165 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:47:32.191 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:47:32.217 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:47:32.246 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:47:32.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:47:32.313 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:47:32.345 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:47:32.345 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:47:32.345 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:47:32.345 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:47:32.345 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:47:32.346 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:47:32.347 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:47:32.347 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:47:32.348 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:47:32.348 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:47:32.348 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:47:32.348 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:47:32.348 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:47:32.348 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:47:32.396 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:47:33.120 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:47:33.121 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T22:47:33.437 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:47:33.438 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:47:34.466 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:47:35.443 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:47:35.621 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:47:36.320 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:47:36.320 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T22:47:37.218 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:47:38.165 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:47:38.165 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:47:38.191 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:47:38.216 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:47:38.248 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:47:38.269 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:47:38.295 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:47:38.321 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:47:38.348 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:47:38.383 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:47:38.409 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:47:38.412 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:47:38.412 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:47:38.412 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:47:38.412 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:47:38.412 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:47:38.412 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:47:38.413 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:47:38.413 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:47:38.413 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:47:38.413 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:47:38.413 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:47:38.413 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:47:38.413 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:47:39.244 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:47:39.244 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:47:39.600 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:47:40.302 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:47:40.303 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T22:47:40.893 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:47:41.245 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:47:43.149 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:47:43.595 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:47:43.895 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:47:43.896 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T22:47:44.084 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:47:44.108 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:47:44.130 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:47:44.153 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:47:44.180 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:47:44.223 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:47:44.258 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:47:44.293 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:47:44.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:47:44.359 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:47:44.396 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:47:44.398 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:47:44.398 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:47:44.398 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:47:44.398 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:47:44.398 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:47:44.398 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:47:44.399 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:47:44.399 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:47:44.399 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:47:44.399 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:47:44.399 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:47:44.399 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:47:44.400 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:47:45.405 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:47:45.405 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:47:46.946 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:47:47.241 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:47:47.406 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:47:47.528 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:47:47.528 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T22:47:49.944 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:47:50.380 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:47:50.707 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:47:51.236 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:47:51.303 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:47:51.306 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:47:51.354 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:47:51.384 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:47:51.393 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:47:51.394 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T22:47:51.412 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:47:51.439 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:47:51.480 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:47:51.514 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:47:51.542 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:47:51.543 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:47:51.543 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:47:51.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:47:51.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:47:51.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:47:51.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:47:51.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:47:51.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:47:51.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:47:51.545 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:47:51.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:47:51.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:47:51.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:47:52.629 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:47:52.629 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:47:53.661 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:47:54.399 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:47:54.630 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:47:54.971 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:47:54.972 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T22:47:56.358 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:47:57.080 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:47:57.112 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:47:57.141 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:47:57.169 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:47:57.199 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:47:57.228 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:47:57.252 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:47:57.280 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:47:57.306 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:47:57.337 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:47:57.363 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:47:57.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:47:57.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:47:57.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:47:57.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:47:57.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:47:57.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:47:57.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:47:57.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:47:57.366 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:47:57.366 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:47:57.366 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:47:57.366 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:47:57.366 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:47:57.460 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:47:58.237 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:47:58.238 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T22:47:58.711 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:47:58.712 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:48:00.653 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:48:00.712 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:48:01.243 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:48:01.244 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T22:48:03.662 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:48:03.696 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:48:03.727 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:48:03.755 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:48:03.783 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:48:03.815 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:48:03.843 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:48:03.867 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:48:03.872 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:48:03.896 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:48:03.923 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:48:03.949 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:48:03.950 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:48:03.950 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:48:03.950 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:48:03.951 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:48:03.951 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:48:03.951 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:48:03.951 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:48:03.951 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:48:03.951 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:48:03.957 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:48:03.957 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:48:03.958 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:48:03.958 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:48:04.573 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:48:04.574 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T22:48:04.773 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:48:04.773 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:48:06.778 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:48:06.790 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:48:07.379 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T22:48:09.803 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:48:09.824 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:48:09.853 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:48:09.874 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:48:09.900 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:48:09.912 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:48:09.913 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:48:09.913 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:48:09.913 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:48:09.913 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:48:09.926 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:48:09.959 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:48:09.987 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:48:10.017 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:48:10.040 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:48:10.076 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:48:10.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:48:10.077 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:48:10.077 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:48:10.077 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:48:10.078 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:48:10.078 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:48:10.078 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:48:10.078 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:48:10.078 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:48:10.078 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:48:10.078 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:48:10.079 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:48:10.227 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:48:10.526 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T22:48:11.428 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:48:11.429 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:48:13.085 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:48:13.085 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:48:13.085 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:48:13.085 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:48:13.086 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:48:13.113 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:48:13.430 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:48:13.629 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T22:48:16.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:48:16.941 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:48:17.473 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T22:48:17.491 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:48:17.688 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:48:17.727 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:48:17.765 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:48:17.798 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:48:17.826 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:48:17.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:48:17.889 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:48:17.923 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:48:17.951 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:48:17.979 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:48:17.981 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:48:17.981 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:48:17.981 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:48:17.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:48:17.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:48:17.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:48:17.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:48:17.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:48:17.990 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:48:17.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:48:17.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:48:17.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:48:18.807 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:48:18.808 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:48:20.147 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:48:20.229 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:48:20.767 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T22:48:20.809 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:48:23.938 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:48:24.212 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:48:24.236 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:48:24.260 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:48:24.283 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:48:24.309 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:48:24.335 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:48:24.361 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:48:24.387 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:48:24.414 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:48:24.441 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:48:24.466 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:48:24.468 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:48:24.469 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:48:24.469 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:48:24.469 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:48:24.469 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:48:24.469 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:48:24.469 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:48:24.470 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:48:24.470 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:48:24.470 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:48:24.470 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:48:24.470 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:48:24.470 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:48:24.486 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:48:24.564 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T22:48:25.556 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:48:25.557 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:48:27.201 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:48:27.201 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:48:27.201 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:48:27.202 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:48:27.225 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:48:27.558 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:48:27.752 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T22:48:30.521 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:48:30.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:48:30.894 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:48:30.918 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:48:30.942 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:48:30.968 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:48:30.995 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:48:31.021 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:48:31.049 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:48:31.095 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:48:31.138 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:48:31.175 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:48:31.178 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:48:31.178 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:48:31.178 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:48:31.179 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:48:31.179 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:48:31.180 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:48:31.180 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:48:31.180 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:48:31.180 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:48:31.180 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:48:31.180 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:48:31.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:48:31.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:48:31.458 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:48:31.813 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:48:31.814 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:48:32.249 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T22:48:33.625 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:48:33.819 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:48:34.395 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:48:35.242 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T22:48:37.497 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:48:37.797 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:48:37.822 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:48:37.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:48:37.886 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:48:37.902 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:48:37.927 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:48:37.955 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:48:37.983 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:48:38.032 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:48:38.072 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:48:38.102 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:48:38.105 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:48:38.105 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:48:38.105 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:48:38.105 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:48:38.106 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:48:38.106 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:48:38.106 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:48:38.106 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:48:38.106 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:48:38.106 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:48:38.106 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:48:38.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:48:38.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:48:38.741 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:48:38.741 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:48:38.809 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:48:38.809 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:48:38.809 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:48:38.809 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:48:39.337 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T22:48:40.305 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:48:40.742 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:48:42.055 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:48:42.056 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:48:42.056 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:48:42.056 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:48:42.056 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:48:42.763 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T22:48:44.501 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:48:44.699 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:48:44.723 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:48:44.746 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:48:44.770 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:48:44.797 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:48:44.823 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:48:44.850 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:48:44.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:48:44.944 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:48:44.981 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:48:45.021 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:48:45.021 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:48:45.021 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:48:45.021 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:48:45.021 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:48:45.021 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:48:45.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:48:45.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:48:45.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:48:45.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:48:45.022 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:48:45.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:48:45.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:48:45.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:48:45.736 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:48:45.736 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:48:45.955 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:48:46.492 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T22:48:47.116 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:48:47.738 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:48:48.953 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:48:49.895 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:48:49.895 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T22:48:51.154 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:48:51.529 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:48:51.555 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:48:51.592 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:48:51.629 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:48:51.659 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:48:51.688 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:48:51.716 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:48:51.743 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:48:51.772 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:48:51.816 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:48:51.854 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:48:51.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:48:51.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:48:51.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:48:51.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:48:51.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:48:51.858 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:48:51.858 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:48:51.858 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:48:51.858 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:48:51.858 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:48:51.858 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:48:51.859 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:48:51.859 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:48:52.643 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:48:52.643 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:48:53.058 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:48:53.677 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:48:53.677 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T22:48:54.040 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:48:54.645 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:48:56.919 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:48:57.507 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:48:57.677 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:48:57.678 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T22:48:58.655 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:48:58.680 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:48:58.705 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:48:58.728 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:48:58.759 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:48:58.786 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:48:58.811 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:48:58.838 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:48:58.865 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:48:58.897 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:48:58.924 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:48:58.925 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:48:58.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:48:58.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:48:58.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:48:58.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:48:58.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:48:58.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:48:58.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:48:58.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:48:58.927 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:48:58.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:48:58.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:48:58.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:48:59.801 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:48:59.802 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:49:00.257 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:49:00.343 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:49:00.843 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:49:00.844 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T22:49:01.803 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:49:03.520 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:49:03.664 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:49:04.521 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:49:04.521 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T22:49:04.776 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:49:04.799 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:49:04.823 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:49:04.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:49:04.872 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:49:04.899 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:49:04.925 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:49:04.952 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:49:04.981 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:49:05.019 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:49:05.052 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:49:05.054 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:49:05.054 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:49:05.054 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:49:05.054 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:49:05.054 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:49:05.054 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:49:05.054 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:49:05.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:49:05.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:49:05.055 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:49:05.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:49:05.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:49:05.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:49:05.637 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:49:05.637 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:49:06.212 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:49:06.586 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:49:07.249 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:49:07.250 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T22:49:07.639 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:49:08.638 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:49:10.041 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:49:10.654 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:49:10.654 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T22:49:10.730 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:49:10.753 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:49:10.777 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:49:10.802 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:49:10.828 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:49:10.854 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:49:10.883 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:49:10.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:49:10.935 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:49:10.962 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:49:10.988 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:49:10.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:49:10.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:49:10.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:49:10.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:49:10.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:49:10.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:49:10.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:49:10.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:49:10.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:49:10.991 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:49:10.991 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:49:10.991 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:49:10.991 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:49:11.227 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:49:11.679 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:49:11.679 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:49:12.974 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:49:13.595 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:49:13.595 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T22:49:13.680 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:49:14.878 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:49:16.183 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:49:16.727 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:49:16.728 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T22:49:16.753 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:49:17.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:49:17.649 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:49:17.677 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:49:17.708 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:49:17.748 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:49:17.757 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:49:17.789 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:49:17.826 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:49:17.859 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:49:17.886 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:49:17.913 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:49:17.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:49:17.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:49:17.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:49:17.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:49:17.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:49:17.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:49:17.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:49:17.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:49:17.916 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:49:17.916 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:49:17.916 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:49:17.916 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:49:17.916 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:49:18.518 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:49:18.519 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:49:19.203 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:49:19.841 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:49:19.841 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T22:49:20.519 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:49:20.645 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:49:22.253 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:49:22.995 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:49:22.996 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T22:49:23.431 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:49:23.456 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:49:23.531 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:49:23.545 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:49:23.551 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:49:23.582 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:49:23.611 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:49:23.637 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:49:23.671 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:49:23.700 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:49:23.728 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:49:23.776 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:49:23.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:49:23.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:49:23.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:49:23.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:49:23.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:49:23.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:49:23.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:49:23.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:49:23.779 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:49:23.779 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:49:23.779 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:49:23.779 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:49:23.779 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:49:24.934 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:49:24.934 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:49:25.699 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:49:26.186 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:49:26.187 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T22:49:26.516 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:49:26.935 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:49:28.381 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:49:29.129 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:49:29.129 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T22:49:29.185 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:49:29.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:49:29.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:49:29.580 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:49:29.603 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:49:29.629 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:49:29.655 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:49:29.681 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:49:29.708 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:49:29.734 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:49:29.775 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:49:29.788 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:49:29.789 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:49:29.789 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:49:29.790 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:49:29.790 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:49:29.790 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:49:29.790 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:49:29.790 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:49:29.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:49:29.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:49:29.791 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:49:29.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:49:29.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:49:29.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:49:30.476 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:49:30.476 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:49:31.988 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:49:32.454 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:49:32.455 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T22:49:32.477 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:49:32.530 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:49:34.988 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:49:35.002 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:49:35.528 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:49:35.534 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:49:35.535 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T22:49:36.400 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:49:36.444 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:49:36.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:49:36.523 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:49:36.556 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:49:36.593 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:49:36.618 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:49:36.645 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:49:36.673 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:49:36.699 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:49:36.701 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:49:36.701 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:49:36.701 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:49:36.702 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:49:36.702 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:49:36.702 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:49:36.702 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:49:36.702 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:49:36.702 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:49:36.702 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:49:36.703 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:49:36.703 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:49:36.703 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:49:37.451 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:49:37.452 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:49:37.978 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:49:38.468 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:49:38.469 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T22:49:38.536 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:49:39.454 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:49:40.903 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:49:41.014 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:49:41.595 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:49:41.596 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T22:49:41.863 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:49:41.886 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:49:41.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:49:41.932 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:49:41.958 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:49:41.985 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:49:42.010 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:49:42.042 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:49:42.073 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:49:42.101 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:49:42.128 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:49:42.130 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:49:42.130 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:49:42.130 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:49:42.130 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:49:42.131 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:49:42.131 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:49:42.131 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:49:42.131 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:49:42.131 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:49:42.131 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:49:42.131 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:49:42.132 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:49:42.132 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:49:42.792 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:49:42.792 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:49:44.056 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:49:44.545 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:49:44.566 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:49:44.566 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T22:49:44.793 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:49:46.980 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:49:47.199 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:49:47.435 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:49:47.435 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T22:49:47.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:49:47.479 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:49:47.500 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:49:47.524 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:49:47.551 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:49:47.577 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:49:47.603 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:49:47.629 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:49:47.658 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:49:47.684 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:49:47.710 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:49:47.711 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:49:47.712 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:49:47.712 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:49:47.712 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:49:47.712 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:49:47.712 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:49:47.712 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:49:47.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:49:47.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:49:47.713 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:49:47.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:49:47.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:49:47.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:49:48.372 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:49:48.372 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:49:49.861 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:49:50.343 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:49:50.344 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T22:49:50.373 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:49:50.550 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:49:52.844 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:49:52.867 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:49:52.893 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:49:52.921 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:49:52.947 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:49:52.974 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:49:52.994 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:49:53.008 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:49:53.034 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:49:53.061 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:49:53.088 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:49:53.114 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:49:53.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:49:53.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:49:53.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:49:53.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:49:53.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:49:53.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:49:53.117 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:49:53.117 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:49:53.117 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:49:53.117 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:49:53.117 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:49:53.117 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:49:53.117 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:49:53.484 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:49:53.561 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:49:53.561 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T22:49:53.751 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:49:53.751 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:49:55.752 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:49:55.985 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:49:55.985 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:49:55.985 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:49:55.985 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:49:56.252 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:49:56.609 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T22:49:58.529 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:49:58.564 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:49:58.594 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:49:58.631 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:49:58.648 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:49:58.674 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:49:58.705 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:49:58.726 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:49:58.753 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:49:58.780 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:49:58.807 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:49:58.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:49:58.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:49:58.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:49:58.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:49:58.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:49:58.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:49:58.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:49:58.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:49:58.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:49:58.811 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:49:58.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:49:58.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:49:58.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:49:59.052 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:49:59.061 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:49:59.601 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:49:59.601 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:49:59.880 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T22:50:01.602 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:50:02.220 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:50:02.557 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:50:02.760 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T22:50:04.211 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:50:04.235 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:50:04.258 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:50:04.285 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:50:04.311 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:50:04.336 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:50:04.362 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:50:04.389 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:50:04.414 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:50:04.441 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:50:04.468 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:50:04.471 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:50:04.471 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:50:04.472 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:50:04.472 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:50:04.472 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:50:04.472 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:50:04.472 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:50:04.472 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:50:04.473 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:50:04.473 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:50:04.473 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:50:04.473 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:50:04.473 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:50:05.103 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:50:05.104 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:50:05.374 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:50:05.467 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:50:05.467 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:50:05.468 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:50:05.468 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:50:05.966 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T22:50:07.105 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:50:08.382 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:50:08.382 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:50:08.382 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:50:08.382 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:50:08.383 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:50:08.556 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:50:09.220 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T22:50:09.963 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:50:10.817 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:50:10.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:50:10.914 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:50:10.943 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:50:10.981 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:50:11.016 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:50:11.046 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:50:11.081 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:50:11.106 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:50:11.135 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:50:11.136 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:50:11.137 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:50:11.137 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:50:11.137 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:50:11.137 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:50:11.137 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:50:11.138 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:50:11.138 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:50:11.138 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:50:11.138 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:50:11.139 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:50:11.139 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:50:11.139 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:50:11.164 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:50:11.409 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:50:11.750 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:50:11.751 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:50:11.955 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T22:50:13.752 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:50:14.117 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:50:14.833 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T22:50:14.875 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:50:16.584 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:50:16.623 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:50:16.650 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:50:16.677 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:50:16.704 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:50:16.732 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:50:16.760 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:50:16.784 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:50:16.811 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:50:16.841 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:50:16.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:50:16.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:50:16.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:50:16.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:50:16.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:50:16.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:50:16.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:50:16.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:50:16.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:50:16.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:50:16.871 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:50:16.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:50:16.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:50:16.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:50:17.371 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:50:17.372 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:50:17.405 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:50:17.405 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:50:17.405 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:50:17.406 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:50:17.406 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:50:17.718 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:50:17.891 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T22:50:19.373 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:50:20.201 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:50:20.201 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:50:20.201 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:50:20.201 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:50:20.201 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:50:20.265 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:50:20.865 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T22:50:22.276 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:50:22.299 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:50:22.324 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:50:22.347 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:50:22.373 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:50:22.402 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:50:22.428 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:50:22.454 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:50:22.481 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:50:22.509 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:50:22.539 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:50:22.539 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:50:22.540 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:50:22.540 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:50:22.541 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:50:22.542 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:50:22.542 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:50:22.542 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:50:22.542 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:50:22.542 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:50:22.543 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:50:22.543 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:50:22.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:50:22.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:50:23.013 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:50:23.029 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:50:23.029 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:50:23.126 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:50:23.485 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T22:50:25.030 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:50:25.734 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:50:25.874 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:50:26.293 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T22:50:27.949 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:50:27.985 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:50:28.004 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:50:28.029 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:50:28.056 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:50:28.082 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:50:28.107 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:50:28.133 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:50:28.161 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:50:28.187 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:50:28.229 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:50:28.230 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:50:28.231 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:50:28.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:50:28.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:50:28.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:50:28.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:50:28.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:50:28.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:50:28.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:50:28.233 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:50:28.241 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:50:28.241 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:50:28.241 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:50:28.694 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:50:28.695 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:50:28.695 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:50:28.695 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:50:28.695 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:50:28.715 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:50:28.715 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:50:28.802 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:50:29.187 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T22:50:30.725 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:50:31.390 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:50:32.047 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:50:32.047 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T22:50:32.134 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:50:33.560 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:50:33.595 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:50:33.628 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:50:33.660 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:50:33.697 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:50:33.721 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:50:33.749 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:50:33.777 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:50:33.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:50:33.836 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:50:33.863 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:50:33.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:50:33.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:50:33.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:50:33.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:50:33.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:50:33.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:50:33.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:50:33.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:50:33.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:50:33.866 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:50:33.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:50:33.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:50:33.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:50:34.473 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:50:34.473 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:50:34.742 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:50:34.952 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:50:35.226 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:50:35.227 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T22:50:36.474 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:50:37.569 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:50:37.611 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:50:38.273 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:50:38.273 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T22:50:39.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:50:39.540 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:50:39.566 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:50:39.590 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:50:39.622 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:50:39.648 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:50:39.684 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:50:39.704 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:50:39.727 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:50:39.755 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:50:39.785 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:50:39.785 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:50:39.785 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:50:39.785 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:50:39.785 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:50:39.785 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:50:39.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:50:39.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:50:39.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:50:39.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:50:39.786 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:50:39.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:50:39.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:50:39.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:50:40.385 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:50:40.408 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:50:40.409 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:50:40.643 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:50:40.809 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:50:40.809 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T22:50:42.410 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:50:43.239 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:50:43.564 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:50:43.915 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:50:43.916 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T22:50:45.249 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:50:45.275 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:50:45.302 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:50:45.327 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:50:45.353 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:50:45.382 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:50:45.406 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:50:45.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:50:45.484 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:50:45.522 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:50:45.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:50:45.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:50:45.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:50:45.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:50:45.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:50:45.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:50:45.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:50:45.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:50:45.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:50:45.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:50:45.560 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:50:45.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:50:45.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:50:45.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:50:46.180 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:50:46.180 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:50:46.395 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:50:46.456 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:50:46.933 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:50:46.933 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T22:50:48.181 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:50:49.325 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:50:49.591 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:50:50.270 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:50:50.271 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T22:50:51.005 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:50:51.040 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:50:51.070 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:50:51.094 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:50:51.120 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:50:51.147 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:50:51.173 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:50:51.219 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:50:51.258 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:50:51.293 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:50:51.326 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:50:51.327 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:50:51.327 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:50:51.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:50:51.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:50:51.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:50:51.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:50:51.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:50:51.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:50:51.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:50:51.329 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:50:51.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:50:51.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:50:51.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:50:52.070 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:50:52.070 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:50:52.549 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:50:52.629 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:50:53.199 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:50:53.200 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T22:50:54.071 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:50:55.850 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:50:56.263 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:50:56.470 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:50:56.471 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T22:50:57.181 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:50:57.210 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:50:57.235 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:50:57.260 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:50:57.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:50:57.313 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:50:57.338 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:50:57.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:50:57.403 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:50:57.423 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:50:57.449 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:50:57.451 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:50:57.452 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:50:57.452 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:50:57.452 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:50:57.452 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:50:57.452 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:50:57.452 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:50:57.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:50:57.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:50:57.453 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:50:57.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:50:57.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:50:57.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:50:58.213 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:50:58.213 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:50:58.768 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:50:58.930 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:50:59.261 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:50:59.262 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T22:51:00.214 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:51:01.905 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:51:02.451 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:51:02.451 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T22:51:02.573 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:51:02.827 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:51:02.850 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:51:02.873 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:51:02.895 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:51:02.923 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:51:02.967 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:51:03.002 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:51:03.035 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:51:03.089 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:51:03.129 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:51:03.167 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:51:03.170 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:51:03.170 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:51:03.170 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:51:03.171 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:51:03.171 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:51:03.171 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:51:03.171 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:51:03.171 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:51:03.171 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:51:03.171 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:51:03.172 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:51:03.172 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:51:03.172 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:51:03.709 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:51:03.710 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:51:04.704 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:51:05.232 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:51:05.233 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T22:51:05.287 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:51:05.711 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:51:08.330 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:51:08.611 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:51:08.844 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:51:08.844 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T22:51:08.845 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:51:08.879 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:51:08.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:51:08.935 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:51:08.963 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:51:08.989 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:51:09.014 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:51:09.040 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:51:09.066 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:51:09.093 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:51:09.119 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:51:09.120 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:51:09.120 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:51:09.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:51:09.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:51:09.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:51:09.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:51:09.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:51:09.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:51:09.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:51:09.122 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:51:09.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:51:09.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:51:09.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:51:09.748 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:51:09.749 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:51:11.161 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:51:11.220 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:51:11.686 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:51:11.686 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T22:51:11.749 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:51:14.083 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:51:14.637 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:51:15.151 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:51:15.179 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:51:15.192 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:51:15.193 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T22:51:15.203 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:51:15.229 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:51:15.255 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:51:15.294 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:51:15.319 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:51:15.346 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:51:15.373 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:51:15.400 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:51:15.427 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:51:15.428 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:51:15.429 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:51:15.429 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:51:15.429 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:51:15.430 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:51:15.430 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:51:15.430 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:51:15.430 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:51:15.430 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:51:15.431 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:51:15.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:51:15.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:51:15.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:51:16.053 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:51:16.053 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:51:16.692 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:51:17.259 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:51:17.779 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:51:17.779 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T22:51:18.056 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:51:19.901 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:51:20.268 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:51:20.549 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:51:20.549 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T22:51:20.793 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:51:20.816 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:51:20.840 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:51:20.864 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:51:20.893 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:51:20.917 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:51:20.948 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:51:20.981 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:51:21.009 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:51:21.060 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:51:21.064 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:51:21.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:51:21.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:51:21.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:51:21.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:51:21.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:51:21.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:51:21.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:51:21.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:51:21.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:51:21.067 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:51:21.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:51:21.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:51:21.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:51:21.762 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:51:21.762 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:51:22.697 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:51:23.002 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:51:23.135 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:51:23.135 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T22:51:23.763 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:51:25.916 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:51:26.281 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:51:26.550 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:51:26.551 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T22:51:26.986 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:51:27.849 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:51:27.887 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:51:27.920 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:51:27.956 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:51:27.987 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:51:28.015 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:51:28.042 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:51:28.069 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:51:28.096 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:51:28.122 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:51:28.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:51:28.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:51:28.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:51:28.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:51:28.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:51:28.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:51:28.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:51:28.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:51:28.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:51:28.125 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:51:28.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:51:28.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:51:28.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:51:28.710 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:51:28.764 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:51:28.765 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:51:28.983 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:51:29.122 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:51:29.123 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T22:51:30.766 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:51:31.583 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:51:31.721 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:51:32.647 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:51:32.648 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T22:51:33.587 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:51:34.181 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:51:34.445 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:51:34.470 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:51:34.496 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:51:34.526 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:51:34.552 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:51:34.579 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:51:34.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:51:34.656 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:51:34.702 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:51:34.747 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:51:34.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:51:34.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:51:34.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:51:34.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:51:34.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:51:34.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:51:34.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:51:34.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:51:34.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:51:34.750 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:51:34.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:51:34.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:51:34.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:51:34.785 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:51:35.382 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:51:35.383 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:51:35.586 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T22:51:37.383 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:51:37.591 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:51:37.729 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:51:38.610 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T22:51:39.753 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:51:39.775 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:51:39.811 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:51:39.843 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:51:39.879 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:51:39.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:51:39.938 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:51:39.965 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:51:39.992 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:51:40.019 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:51:40.046 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:51:40.048 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:51:40.048 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:51:40.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:51:40.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:51:40.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:51:40.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:51:40.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:51:40.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:51:40.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:51:40.050 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:51:40.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:51:40.051 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:51:40.051 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:51:40.536 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:51:40.536 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:51:40.604 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:51:40.957 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:51:40.958 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:51:40.958 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:51:40.958 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:51:40.958 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:51:41.645 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T22:51:42.538 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:51:43.613 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:51:44.184 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:51:44.184 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:51:44.185 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:51:44.185 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:51:44.950 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T22:51:45.337 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:51:45.357 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:51:45.379 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:51:45.402 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:51:45.429 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:51:45.454 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:51:45.481 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:51:45.508 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:51:45.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:51:45.562 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:51:45.601 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:51:45.602 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:51:45.602 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:51:45.603 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:51:45.603 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:51:45.603 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:51:45.603 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:51:45.603 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:51:45.603 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:51:45.603 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:51:45.604 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:51:45.604 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:51:45.604 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:51:45.604 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:51:46.467 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:51:46.524 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:51:46.524 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:51:47.080 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:51:47.624 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T22:51:48.525 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:51:49.644 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:51:49.886 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:51:50.833 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T22:51:51.425 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:51:51.453 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:51:51.487 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:51:51.514 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:51:51.552 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:51:51.579 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:51:51.613 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:51:51.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:51:51.666 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:51:51.692 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:51:51.718 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:51:51.720 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:51:51.720 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:51:51.720 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:51:51.721 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:51:51.721 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:51:51.721 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:51:51.721 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:51:51.721 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:51:51.721 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:51:51.721 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:51:51.722 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:51:51.722 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:51:51.722 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:51:52.417 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:51:52.485 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:51:52.486 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:51:53.069 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:51:53.069 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:51:53.069 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:51:53.069 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:51:53.616 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T22:51:54.487 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:51:55.279 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:51:56.128 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:51:56.129 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:51:56.129 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:51:56.129 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:51:56.129 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:51:56.855 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T22:51:57.247 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:51:57.276 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:51:57.292 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:51:57.317 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:51:57.343 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:51:57.368 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:51:57.393 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:51:57.419 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:51:57.451 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:51:57.487 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:51:57.506 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:51:57.507 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:51:57.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:51:57.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:51:57.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:51:57.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:51:57.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:51:57.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:51:57.509 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:51:57.509 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:51:57.509 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:51:57.509 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:51:57.509 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:51:57.509 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:51:57.966 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:51:58.022 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:51:58.022 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:51:59.200 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:51:59.838 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T22:52:00.023 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:52:00.572 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:52:02.539 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:52:02.821 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:52:02.844 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:52:02.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:52:02.888 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:52:02.916 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:52:02.963 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:52:03.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:52:03.040 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:52:03.064 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:52:03.094 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:52:03.107 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T22:52:03.122 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:52:03.123 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:52:03.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:52:03.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:52:03.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:52:03.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:52:03.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:52:03.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:52:03.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:52:03.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:52:03.125 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:52:03.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:52:03.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:52:03.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:52:03.722 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:52:03.722 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:52:03.751 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:52:05.687 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:52:05.688 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:52:05.688 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:52:05.688 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:52:05.688 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:52:05.724 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:52:06.369 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T22:52:06.666 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:52:08.404 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:52:08.991 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:52:08.991 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:52:08.991 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:52:08.991 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:52:08.992 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:52:09.267 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:52:09.319 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:52:09.353 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:52:09.387 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:52:09.423 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:52:09.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:52:09.469 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T22:52:09.494 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:52:09.533 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:52:09.569 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:52:09.595 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:52:09.602 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:52:09.603 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:52:09.604 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:52:09.604 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:52:09.604 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:52:09.604 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:52:09.605 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:52:09.605 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:52:09.605 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:52:09.605 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:52:09.605 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:52:09.606 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:52:09.606 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:52:09.606 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:52:10.265 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:52:10.265 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:52:11.958 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:52:12.266 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:52:12.464 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:52:12.464 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T22:52:12.691 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:52:14.864 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:52:14.887 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:52:14.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:52:14.934 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:52:14.961 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:52:14.988 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:52:15.014 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:52:15.040 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:52:15.066 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:52:15.092 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:52:15.112 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:52:15.119 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:52:15.120 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:52:15.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:52:15.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:52:15.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:52:15.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:52:15.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:52:15.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:52:15.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:52:15.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:52:15.122 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:52:15.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:52:15.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:52:15.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:52:15.552 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:52:15.766 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:52:15.767 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T22:52:15.906 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:52:15.906 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:52:17.907 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:52:18.016 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:52:18.509 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:52:18.509 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T22:52:18.657 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:52:20.592 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:52:20.617 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:52:20.648 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:52:20.676 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:52:20.702 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:52:20.746 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:52:20.782 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:52:20.816 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:52:20.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:52:20.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:52:20.926 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:52:20.929 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:52:20.929 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:52:20.929 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:52:20.929 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:52:20.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:52:20.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:52:20.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:52:20.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:52:20.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:52:20.930 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:52:20.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:52:20.931 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:52:20.931 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:52:21.384 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:52:21.414 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:52:21.426 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:52:21.428 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:52:21.844 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:52:21.845 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T22:52:23.428 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:52:24.276 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:52:24.469 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:52:25.438 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:52:25.439 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T22:52:25.975 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:52:26.000 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:52:26.024 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:52:26.048 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:52:26.076 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:52:26.105 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:52:26.131 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:52:26.158 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:52:26.184 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:52:26.229 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:52:26.256 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:52:26.257 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:52:26.257 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:52:26.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:52:26.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:52:26.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:52:26.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:52:26.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:52:26.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:52:26.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:52:26.259 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:52:26.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:52:26.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:52:26.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:52:27.007 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:52:27.008 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:52:27.011 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:52:27.524 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:52:28.065 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:52:28.065 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T22:52:29.009 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:52:29.660 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:52:30.819 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:52:31.405 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:52:31.406 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T22:52:31.755 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:52:31.777 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:52:31.799 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:52:31.824 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:52:31.851 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:52:31.877 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:52:31.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:52:31.930 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:52:31.957 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:52:31.984 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:52:32.021 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:52:32.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:52:32.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:52:32.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:52:32.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:52:32.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:52:32.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:52:32.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:52:32.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:52:32.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:52:32.023 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:52:32.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:52:32.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:52:32.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:52:32.394 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:52:32.737 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:52:32.737 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:52:33.966 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:52:34.588 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:52:34.588 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T22:52:34.739 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:52:35.042 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:52:37.038 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:52:37.545 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:52:37.601 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:52:37.602 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T22:52:37.699 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:52:38.400 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:52:38.446 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:52:38.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:52:38.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:52:38.543 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:52:38.575 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:52:38.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:52:38.637 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:52:38.665 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:52:38.692 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:52:38.693 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:52:38.693 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:52:38.694 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:52:38.694 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:52:38.694 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:52:38.694 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:52:38.694 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:52:38.694 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:52:38.694 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:52:38.695 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:52:38.695 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:52:38.695 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:52:38.695 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:52:39.491 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:52:39.492 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:52:40.122 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:52:40.581 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:52:40.774 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:52:40.774 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T22:52:41.493 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:52:42.920 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:52:43.234 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:52:43.751 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:52:43.751 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T22:52:44.146 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:52:44.171 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:52:44.195 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:52:44.220 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:52:44.247 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:52:44.275 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:52:44.302 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:52:44.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:52:44.370 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:52:44.397 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:52:44.429 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:52:44.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:52:44.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:52:44.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:52:44.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:52:44.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:52:44.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:52:44.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:52:44.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:52:44.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:52:44.433 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:52:44.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:52:44.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:52:44.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:52:45.005 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:52:45.006 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:52:46.424 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:52:46.664 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:52:46.876 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:52:46.876 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T22:52:47.007 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:52:49.075 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:52:49.185 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:52:49.285 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:52:49.309 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:52:49.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:52:49.356 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:52:49.382 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:52:49.409 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:52:49.436 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:52:49.462 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:52:49.488 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:52:49.514 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:52:49.545 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:52:49.548 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:52:49.548 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:52:49.548 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:52:49.548 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:52:49.549 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:52:49.550 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:52:49.550 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:52:49.550 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:52:49.550 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:52:49.550 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:52:49.550 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:52:49.551 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:52:49.551 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:52:49.563 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:52:49.564 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T22:52:50.288 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:52:50.288 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:52:52.136 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:52:52.289 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:52:52.635 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:52:52.636 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T22:52:52.692 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:52:54.723 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:52:54.749 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:52:54.774 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:52:54.799 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:52:54.823 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:52:54.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:52:54.908 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:52:54.943 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:52:54.961 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:52:55.002 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:52:55.027 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:52:55.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:52:55.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:52:55.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:52:55.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:52:55.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:52:55.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:52:55.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:52:55.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:52:55.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:52:55.029 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:52:55.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:52:55.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:52:55.030 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:52:55.149 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:52:55.518 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:52:55.657 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:52:55.657 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:52:55.874 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:52:55.875 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T22:52:57.658 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:52:57.968 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:52:58.561 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:52:58.562 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T22:52:58.667 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:53:00.698 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:53:00.726 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:53:00.757 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:53:00.787 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:53:00.818 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:53:00.853 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:53:00.885 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:53:00.914 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:53:00.942 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:53:00.949 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:53:00.984 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:53:01.015 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:53:01.017 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:53:01.017 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:53:01.017 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:53:01.018 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:53:01.018 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:53:01.018 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:53:01.018 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:53:01.018 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:53:01.018 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:53:01.018 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:53:01.018 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:53:01.019 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:53:01.019 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:53:01.502 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:53:01.541 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:53:01.542 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T22:53:01.711 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:53:01.711 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:53:03.712 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:53:03.751 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:53:04.209 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:53:04.209 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T22:53:04.467 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:53:06.420 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:53:06.665 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:53:07.122 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:53:07.181 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:53:07.181 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T22:53:07.276 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:53:07.307 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:53:07.335 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:53:07.362 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:53:07.399 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:53:07.448 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:53:07.473 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:53:07.511 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:53:07.541 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:53:07.570 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:53:07.571 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:53:07.571 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:53:07.572 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:53:07.572 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:53:07.572 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:53:07.572 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:53:07.572 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:53:07.572 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:53:07.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:53:07.573 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:53:07.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:53:07.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:53:07.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:53:08.313 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:53:08.313 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:53:09.595 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:53:10.169 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:53:10.169 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T22:53:10.314 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:53:10.696 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:53:12.742 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:53:13.218 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:53:13.218 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T22:53:13.324 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:53:13.355 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:53:13.383 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:53:13.409 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:53:13.437 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:53:13.463 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:53:13.486 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:53:13.498 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:53:13.518 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:53:13.542 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:53:13.570 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:53:13.595 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:53:13.597 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:53:13.597 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:53:13.597 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:53:13.597 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:53:13.597 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:53:13.598 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:53:13.598 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:53:13.598 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:53:13.598 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:53:13.599 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:53:13.599 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:53:13.599 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:53:13.599 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:53:14.363 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:53:14.363 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:53:15.666 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:53:15.667 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:53:15.667 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:53:15.667 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:53:16.257 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T22:53:16.364 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:53:16.886 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:53:18.882 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:53:18.882 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:53:18.883 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:53:18.883 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:53:18.883 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:53:19.160 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:53:19.187 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:53:19.213 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:53:19.238 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:53:19.264 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:53:19.290 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:53:19.316 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:53:19.343 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:53:19.372 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:53:19.398 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:53:19.436 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:53:19.438 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:53:19.438 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:53:19.438 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:53:19.438 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:53:19.438 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:53:19.438 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:53:19.439 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:53:19.439 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:53:19.439 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:53:19.439 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:53:19.439 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:53:19.439 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:53:19.450 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T22:53:19.813 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:53:19.903 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:53:19.904 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:53:21.670 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:53:21.904 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:53:22.289 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T22:53:22.682 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:53:24.546 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:53:24.585 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:53:24.613 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:53:24.646 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:53:24.668 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:53:24.695 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:53:24.721 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:53:24.747 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:53:24.773 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:53:24.782 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:53:24.782 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:53:24.782 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:53:24.782 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:53:24.802 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:53:24.828 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:53:24.830 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:53:24.830 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:53:24.830 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:53:24.830 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:53:24.830 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:53:24.830 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:53:24.830 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:53:24.831 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:53:24.831 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:53:24.831 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:53:24.831 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:53:24.831 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:53:24.831 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:53:25.347 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T22:53:25.502 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:53:25.503 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:53:25.514 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:53:27.504 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:53:27.601 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:53:27.602 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:53:27.602 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:53:27.602 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:53:27.602 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:53:28.276 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T22:53:28.712 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:53:30.394 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:53:30.419 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:53:30.443 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:53:30.467 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:53:30.492 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:53:30.519 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:53:30.554 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:53:30.580 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:53:30.609 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:53:30.637 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:53:30.663 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:53:30.664 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:53:30.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:53:30.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:53:30.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:53:30.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:53:30.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:53:30.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:53:30.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:53:30.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:53:30.666 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:53:30.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:53:30.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:53:30.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:53:30.711 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:53:31.309 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:53:31.309 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:53:31.464 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T22:53:31.612 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:53:33.310 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:53:33.742 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:53:34.382 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T22:53:34.742 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:53:36.559 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:53:36.582 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:53:36.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:53:36.629 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:53:36.658 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:53:36.685 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:53:36.711 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:53:36.737 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:53:36.764 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:53:36.790 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:53:36.819 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:53:36.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:53:36.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:53:36.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:53:36.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:53:36.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:53:36.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:53:36.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:53:36.822 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:53:36.822 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:53:36.822 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:53:36.822 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:53:36.822 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:53:36.822 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:53:36.853 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:53:36.853 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:53:36.853 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:53:36.853 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:53:36.854 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:53:37.454 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:53:37.454 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:53:37.638 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T22:53:37.693 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:53:39.455 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:53:40.297 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:53:40.297 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:53:40.297 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:53:40.297 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:53:40.298 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:53:40.298 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:53:41.032 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T22:53:42.263 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:53:42.297 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:53:42.349 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:53:42.390 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:53:42.426 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:53:42.461 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:53:42.494 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:53:42.544 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:53:42.583 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:53:42.617 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:53:42.650 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:53:42.651 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:53:42.652 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:53:42.652 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:53:42.652 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:53:42.652 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:53:42.652 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:53:42.653 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:53:42.653 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:53:42.653 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:53:42.653 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:53:42.653 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:53:42.653 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:53:42.653 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:53:43.326 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:53:43.337 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:53:43.337 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:53:43.436 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:53:43.964 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T22:53:45.340 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:53:46.306 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:53:46.779 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:53:47.586 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T22:53:48.011 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:53:48.038 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:53:48.061 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:53:48.084 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:53:48.111 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:53:48.137 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:53:48.164 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:53:48.194 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:53:48.222 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:53:48.254 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:53:48.280 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:53:48.282 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:53:48.282 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:53:48.282 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:53:48.283 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:53:48.283 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:53:48.283 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:53:48.283 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:53:48.283 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:53:48.283 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:53:48.283 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:53:48.284 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:53:48.284 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:53:48.284 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:53:49.090 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:53:49.158 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:53:49.158 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:53:49.772 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:53:49.772 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:53:49.772 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:53:49.772 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:53:49.773 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:53:50.509 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T22:53:51.159 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:53:51.714 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:53:53.351 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:53:53.871 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:53:53.886 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:53:53.886 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T22:53:54.521 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:53:54.724 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:53:54.753 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:53:54.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:53:54.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:53:54.832 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:53:54.857 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:53:54.885 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:53:54.910 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:53:54.936 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:53:54.962 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:53:54.968 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:53:54.968 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:53:54.968 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:53:54.968 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:53:54.968 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:53:54.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:53:54.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:53:54.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:53:54.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:53:54.969 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:53:54.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:53:54.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:53:54.970 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:53:55.758 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:53:55.759 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:53:56.590 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:53:57.113 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:53:57.114 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T22:53:57.697 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:53:57.762 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:53:59.374 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:54:00.141 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:54:00.142 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T22:54:00.291 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:54:00.553 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:54:00.569 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:54:00.594 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:54:00.617 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:54:00.645 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:54:00.669 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:54:00.695 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:54:00.722 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:54:00.748 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:54:00.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:54:00.803 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:54:00.804 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:54:00.804 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:54:00.805 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:54:00.805 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:54:00.805 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:54:00.805 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:54:00.805 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:54:00.805 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:54:00.805 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:54:00.806 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:54:00.806 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:54:00.806 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:54:00.806 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:54:01.457 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:54:01.457 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:54:02.787 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:54:03.370 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:54:03.371 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T22:54:03.458 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:54:03.732 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:54:05.960 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:54:05.983 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:54:06.007 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:54:06.031 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:54:06.057 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:54:06.084 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:54:06.109 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:54:06.136 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:54:06.159 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:54:06.163 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:54:06.190 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:54:06.216 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:54:06.217 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:54:06.217 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:54:06.217 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:54:06.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:54:06.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:54:06.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:54:06.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:54:06.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:54:06.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:54:06.219 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:54:06.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:54:06.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:54:06.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:54:06.639 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:54:06.802 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:54:06.802 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T22:54:06.945 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:54:06.946 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:54:08.947 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:54:09.000 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:54:09.460 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:54:09.460 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T22:54:10.789 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:54:12.078 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:54:12.101 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:54:12.125 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:54:12.134 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:54:12.161 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:54:12.174 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:54:12.201 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:54:12.227 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:54:12.253 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:54:12.280 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:54:12.311 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:54:12.335 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:54:12.337 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:54:12.337 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:54:12.338 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:54:12.338 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:54:12.338 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:54:12.338 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:54:12.338 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:54:12.338 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:54:12.338 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:54:12.339 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:54:12.339 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:54:12.339 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:54:12.339 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:54:12.610 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:54:12.761 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T22:54:12.977 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:54:12.984 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:54:13.585 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:54:14.906 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:54:14.985 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:54:15.530 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:54:15.530 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T22:54:16.815 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:54:17.695 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:54:18.297 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:54:18.572 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:54:18.621 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:54:18.663 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:54:18.710 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:54:18.762 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:54:18.813 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:54:18.826 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:54:18.827 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T22:54:18.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:54:18.887 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:54:18.920 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:54:18.951 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:54:18.953 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:54:18.953 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:54:18.954 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:54:18.954 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:54:18.954 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:54:18.954 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:54:18.954 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:54:18.954 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:54:18.955 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:54:18.955 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:54:18.955 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:54:18.955 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:54:18.955 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:54:19.555 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:54:19.555 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:54:19.724 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:54:21.515 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:54:21.556 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:54:22.045 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:54:22.045 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T22:54:23.065 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:54:24.304 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:54:24.332 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:54:24.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:54:24.386 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:54:24.414 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:54:24.439 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:54:24.466 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:54:24.503 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:54:24.512 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:54:24.545 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:54:24.580 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:54:24.613 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:54:24.615 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:54:24.615 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:54:24.615 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:54:24.616 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:54:24.616 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:54:24.616 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:54:24.616 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:54:24.616 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:54:24.616 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:54:24.617 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:54:24.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:54:24.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:54:24.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:54:25.122 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:54:25.123 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T22:54:25.313 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:54:25.313 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:54:25.978 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:54:27.315 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:54:27.373 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:54:28.109 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:54:28.110 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T22:54:28.718 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:54:30.387 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:54:30.416 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:54:30.445 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:54:30.468 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:54:30.494 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:54:30.521 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:54:30.549 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:54:30.556 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:54:30.576 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:54:30.600 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:54:30.629 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:54:30.655 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:54:30.660 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:54:30.660 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:54:30.661 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:54:30.661 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:54:30.661 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:54:30.661 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:54:30.661 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:54:30.661 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:54:30.661 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:54:30.662 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:54:30.662 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:54:30.662 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:54:30.662 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:54:31.153 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:54:31.154 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T22:54:31.295 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:54:31.296 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:54:31.647 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:54:33.296 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:54:33.299 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:54:34.085 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:54:34.086 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T22:54:34.797 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:54:36.368 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:54:36.392 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:54:36.418 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:54:36.449 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:54:36.478 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:54:36.505 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:54:36.533 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:54:36.559 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:54:36.586 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:54:36.596 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:54:36.613 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:54:36.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:54:36.641 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:54:36.641 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:54:36.642 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:54:36.642 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:54:36.642 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:54:36.642 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:54:36.642 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:54:36.642 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:54:36.642 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:54:36.643 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:54:36.643 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:54:36.643 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:54:36.643 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:54:37.225 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:54:37.226 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T22:54:37.371 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:54:37.371 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:54:37.583 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:54:39.373 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:54:39.748 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:54:40.316 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:54:40.390 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:54:40.391 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T22:54:42.263 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:54:42.289 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:54:42.322 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:54:42.344 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:54:42.371 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:54:42.397 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:54:42.422 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:54:42.449 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:54:42.476 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:54:42.525 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:54:42.562 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:54:42.564 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:54:42.565 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:54:42.565 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:54:42.565 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:54:42.565 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:54:42.565 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:54:42.567 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:54:42.567 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:54:42.568 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:54:42.568 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:54:42.568 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:54:42.569 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:54:42.569 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:54:42.912 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:54:43.009 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:54:43.102 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:54:43.102 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:54:43.468 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:54:43.469 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T22:54:45.104 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:54:45.815 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:54:46.322 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:54:46.458 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:54:46.458 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T22:54:47.898 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:54:48.764 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:54:48.777 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:54:48.804 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:54:48.840 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:54:48.873 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:54:48.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:54:48.933 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:54:48.968 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:54:49.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:54:49.003 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:54:49.033 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:54:49.062 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:54:49.062 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:54:49.063 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:54:49.063 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:54:49.063 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:54:49.063 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:54:49.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:54:49.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:54:49.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:54:49.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:54:49.064 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:54:49.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:54:49.065 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:54:49.065 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:54:49.277 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:54:49.278 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T22:54:49.797 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:54:49.797 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:54:51.591 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:54:51.798 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:54:52.172 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:54:52.173 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T22:54:52.769 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:54:54.462 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:54:54.487 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:54:54.521 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:54:54.550 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:54:54.581 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:54:54.611 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:54:54.643 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:54:54.649 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:54:54.678 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:54:54.704 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:54:54.732 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:54:54.758 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:54:54.760 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:54:54.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:54:54.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:54:54.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:54:54.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:54:54.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:54:54.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:54:54.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:54:54.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:54:54.762 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:54:54.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:54:54.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:54:54.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:54:55.304 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:54:55.304 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T22:54:55.493 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:54:55.493 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:54:56.040 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:54:57.495 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:54:57.592 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:54:58.306 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T22:54:58.855 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:55:00.555 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:55:00.593 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:55:00.626 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:55:00.655 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:55:00.683 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:55:00.710 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:55:00.737 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:55:00.769 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:55:00.796 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:55:00.823 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:55:00.849 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:55:00.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:55:00.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:55:00.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:55:00.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:55:00.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:55:00.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:55:00.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:55:00.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:55:00.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:55:00.853 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:55:00.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:55:00.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:55:00.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:55:00.854 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:55:00.854 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:55:00.854 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:55:00.854 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:55:00.855 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:55:01.489 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:55:01.489 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:55:01.641 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T22:55:01.695 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:55:03.490 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:55:04.010 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:55:04.011 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:55:04.011 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:55:04.011 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:55:04.011 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:55:04.317 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:55:04.714 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T22:55:05.886 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:55:05.911 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:55:05.934 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:55:05.958 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:55:05.984 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:55:06.010 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:55:06.038 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:55:06.066 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:55:06.096 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:55:06.122 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:55:06.148 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:55:06.150 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:55:06.150 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:55:06.150 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:55:06.150 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:55:06.150 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:55:06.151 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:55:06.151 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:55:06.151 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:55:06.151 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:55:06.151 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:55:06.151 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:55:06.152 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:55:06.152 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:55:06.636 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:55:06.636 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:55:07.035 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:55:07.222 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:55:07.817 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T22:55:08.637 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:55:10.350 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:55:10.366 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:55:11.142 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T22:55:11.801 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:55:11.824 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:55:11.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:55:11.872 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:55:11.898 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:55:11.925 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:55:11.951 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:55:11.977 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:55:12.005 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:55:12.031 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:55:12.058 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:55:12.059 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:55:12.059 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:55:12.059 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:55:12.059 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:55:12.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:55:12.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:55:12.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:55:12.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:55:12.061 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:55:12.061 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:55:12.061 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:55:12.061 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:55:12.061 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:55:12.821 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:55:12.822 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:55:13.383 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:55:13.401 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:55:13.401 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:55:13.401 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:55:13.401 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:55:13.401 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:55:13.943 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T22:55:14.822 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:55:16.507 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:55:16.681 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:55:16.682 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:55:16.682 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:55:16.682 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:55:17.477 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T22:55:17.871 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:55:17.895 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:55:17.917 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:55:17.941 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:55:17.970 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:55:17.998 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:55:18.024 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:55:18.052 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:55:18.093 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:55:18.129 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:55:18.160 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:55:18.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:55:18.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:55:18.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:55:18.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:55:18.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:55:18.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:55:18.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:55:18.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:55:18.161 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:55:18.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:55:18.162 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:55:18.162 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:55:18.811 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:55:18.812 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:55:19.229 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:55:19.620 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:55:20.335 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T22:55:20.813 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:55:22.069 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:55:23.296 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:55:23.392 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:55:23.423 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:55:23.452 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:55:23.480 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:55:23.509 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:55:23.536 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:55:23.562 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:55:23.590 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:55:23.615 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:55:23.664 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:55:23.704 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:55:23.707 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:55:23.707 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:55:23.707 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:55:23.707 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:55:23.707 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:55:23.708 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:55:23.708 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:55:23.708 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:55:23.708 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:55:23.708 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:55:23.708 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:55:23.708 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:55:23.709 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:55:23.810 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T22:55:24.143 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:55:24.144 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:55:24.965 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:55:25.672 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:55:25.673 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:55:25.673 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:55:25.673 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:55:26.145 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:55:26.234 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T22:55:28.748 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:55:28.836 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:55:29.243 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:55:29.243 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:55:29.243 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:55:29.243 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:55:29.243 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:55:29.661 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T22:55:29.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:55:29.737 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:55:29.820 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:55:29.821 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:55:29.841 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:55:29.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:55:29.894 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:55:29.923 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:55:29.952 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:55:29.978 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:55:29.980 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:55:29.980 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:55:29.980 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:55:29.980 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:55:29.981 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:55:29.981 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:55:29.981 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:55:29.981 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:55:29.981 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:55:29.981 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:55:29.981 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:55:29.982 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:55:29.982 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:55:30.521 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:55:30.521 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:55:31.274 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:55:31.730 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:55:32.410 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T22:55:32.522 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:55:34.800 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:55:34.840 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:55:35.488 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:55:35.514 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:55:35.554 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:55:35.588 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:55:35.620 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:55:35.649 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:55:35.677 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:55:35.708 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:55:35.725 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:55:35.725 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T22:55:35.735 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:55:35.777 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:55:35.812 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:55:35.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:55:35.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:55:35.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:55:35.814 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:55:35.814 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:55:35.814 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:55:35.814 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:55:35.814 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:55:35.814 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:55:35.814 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:55:35.814 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:55:35.815 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:55:35.815 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:55:36.280 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:55:36.280 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:55:37.539 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:55:37.770 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:55:38.282 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:55:38.412 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:55:38.412 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T22:55:40.965 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:55:41.033 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:55:41.059 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:55:41.084 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:55:41.108 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:55:41.135 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:55:41.165 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:55:41.191 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:55:41.225 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:55:41.246 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:55:41.273 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:55:41.300 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:55:41.301 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:55:41.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:55:41.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:55:41.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:55:41.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:55:41.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:55:41.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:55:41.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:55:41.303 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:55:41.303 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:55:41.303 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:55:41.303 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:55:41.303 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:55:41.608 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:55:41.895 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:55:41.895 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:55:42.222 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:55:42.222 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T22:55:43.737 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:55:43.896 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:55:44.138 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:55:45.073 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:55:45.074 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T22:55:47.226 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:55:47.255 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:55:47.288 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:55:47.314 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:55:47.335 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:55:47.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:55:47.403 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:55:47.419 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:55:47.448 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:55:47.473 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:55:47.499 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:55:47.501 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:55:47.501 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:55:47.501 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:55:47.501 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:55:47.501 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:55:47.502 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:55:47.502 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:55:47.502 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:55:47.502 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:55:47.502 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:55:47.502 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:55:47.502 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:55:47.503 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:55:47.607 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:55:48.098 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:55:48.099 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:55:48.247 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:55:48.247 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T22:55:50.100 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:55:50.143 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:55:50.729 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:55:50.729 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T22:55:52.252 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:55:52.274 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:55:52.299 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:55:52.323 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:55:52.350 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:55:52.376 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:55:52.409 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:55:52.431 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:55:52.457 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:55:52.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:55:52.511 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:55:52.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:55:52.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:55:52.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:55:52.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:55:52.515 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:55:52.515 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:55:52.515 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:55:52.515 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:55:52.515 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:55:52.515 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:55:52.516 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:55:52.516 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:55:52.516 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:55:53.149 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:55:53.150 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:55:53.365 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:55:53.832 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:55:53.833 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T22:55:55.150 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:55:55.812 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:55:56.370 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:55:56.370 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T22:55:57.393 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:55:57.415 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:55:57.438 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:55:57.461 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:55:57.487 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:55:57.513 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:55:57.539 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:55:57.566 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:55:57.591 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:55:57.619 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:55:57.648 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:55:57.650 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:55:57.650 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:55:57.650 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:55:57.651 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:55:57.651 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:55:57.651 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:55:57.651 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:55:57.651 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:55:57.651 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:55:57.651 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:55:57.652 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:55:57.652 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:55:57.652 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:55:57.760 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:55:58.260 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:55:58.260 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:55:58.690 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:55:59.110 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:55:59.111 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T22:56:00.262 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:56:00.671 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:56:01.014 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:56:01.668 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:56:01.668 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T22:56:03.313 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:56:03.344 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:56:03.376 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:56:03.405 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:56:03.437 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:56:03.464 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:56:03.491 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:56:03.539 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:56:03.578 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:56:03.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:56:03.646 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:56:03.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:56:03.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:56:03.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:56:03.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:56:03.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:56:03.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:56:03.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:56:03.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:56:03.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:56:03.649 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:56:03.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:56:03.650 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:56:03.650 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:56:04.252 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:56:04.326 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:56:04.326 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:56:04.728 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:56:04.729 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T22:56:04.774 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:56:06.327 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:56:06.725 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:56:07.346 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:56:07.483 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:56:07.483 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T22:56:09.404 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:56:09.428 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:56:09.464 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:56:09.495 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:56:09.523 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:56:09.559 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:56:09.578 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:56:09.604 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:56:09.659 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:56:09.701 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:56:09.738 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:56:09.739 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:56:09.740 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:56:09.740 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:56:09.740 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:56:09.740 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:56:09.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:56:09.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:56:09.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:56:09.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:56:09.741 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:56:09.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:56:09.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:56:09.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:56:10.292 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:56:10.431 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:56:10.431 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:56:10.811 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:56:10.811 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T22:56:10.841 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:56:12.433 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:56:12.922 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:56:13.493 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:56:13.529 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:56:13.529 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T22:56:14.786 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:56:14.809 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:56:14.833 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:56:14.856 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:56:14.884 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:56:14.911 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:56:14.937 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:56:14.964 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:56:14.991 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:56:15.019 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:56:15.095 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:56:15.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:56:15.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:56:15.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:56:15.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:56:15.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:56:15.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:56:15.097 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:56:15.097 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:56:15.097 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:56:15.097 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:56:15.097 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:56:15.097 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:56:15.097 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:56:15.985 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:56:15.986 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:56:16.400 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:56:16.993 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:56:16.994 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T22:56:17.778 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:56:17.986 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:56:19.343 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:56:19.970 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:56:19.971 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T22:56:20.543 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:56:20.617 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:56:20.652 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:56:20.688 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:56:20.715 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:56:20.745 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:56:20.771 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:56:20.799 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:56:20.825 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:56:20.857 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:56:20.879 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:56:20.907 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:56:20.909 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:56:20.909 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:56:20.909 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:56:20.909 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:56:20.910 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:56:20.910 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:56:20.910 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:56:20.910 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:56:20.910 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:56:20.910 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:56:20.910 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:56:20.911 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:56:20.911 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:56:21.928 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:56:21.928 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:56:22.563 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:56:23.119 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:56:23.119 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T22:56:23.798 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:56:23.929 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:56:25.414 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:56:26.190 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:56:26.191 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T22:56:26.568 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:56:26.591 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:56:26.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:56:26.638 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:56:26.653 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:56:26.664 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:56:26.699 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:56:26.734 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:56:26.762 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:56:26.789 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:56:26.817 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:56:26.843 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:56:26.845 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:56:26.845 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:56:26.845 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:56:26.845 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:56:26.845 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:56:26.845 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:56:26.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:56:26.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:56:26.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:56:26.846 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:56:26.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:56:26.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:56:26.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:56:27.642 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:56:27.642 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:56:28.735 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:56:29.229 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:56:29.229 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T22:56:29.644 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:56:29.817 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:56:31.686 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:56:32.296 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:56:32.296 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T22:56:32.357 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:56:32.611 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:56:32.633 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:56:32.656 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:56:32.680 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:56:32.707 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:56:32.734 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:56:32.773 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:56:32.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:56:32.834 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:56:32.863 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:56:32.893 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:56:32.898 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:56:32.899 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:56:32.899 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:56:32.900 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:56:32.900 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:56:32.900 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:56:32.900 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:56:32.901 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:56:32.901 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:56:32.902 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:56:32.902 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:56:32.902 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:56:32.902 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:56:33.653 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:56:33.653 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:56:34.791 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:56:35.283 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:56:35.284 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T22:56:35.654 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:56:36.341 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:56:37.949 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:56:37.949 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:56:37.950 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:56:37.950 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:56:38.190 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:56:38.608 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T22:56:39.046 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:56:39.084 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:56:39.118 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:56:39.153 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:56:39.179 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:56:39.207 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:56:39.215 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:56:39.235 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:56:39.271 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:56:39.315 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:56:39.351 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:56:39.353 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:56:39.353 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:56:39.353 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:56:39.354 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:56:39.354 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:56:39.354 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:56:39.354 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:56:39.355 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:56:39.355 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:56:39.355 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:56:39.355 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:56:39.355 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:56:39.355 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:56:39.973 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:56:39.974 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:56:40.970 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:56:41.758 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T22:56:41.975 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:56:43.071 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:56:44.439 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:56:44.891 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:56:44.927 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T22:56:45.748 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:56:45.772 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:56:45.799 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:56:45.825 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:56:45.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:56:45.879 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:56:45.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:56:45.931 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:56:45.959 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:56:45.961 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:56:45.986 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:56:45.988 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:56:45.988 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:56:45.988 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:56:45.988 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:56:45.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:56:45.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:56:45.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:56:45.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:56:45.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:56:45.989 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:56:45.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:56:45.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:56:45.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:56:46.507 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:56:46.508 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:56:47.225 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:56:47.231 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:56:47.231 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:56:47.232 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:56:47.920 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T22:56:48.509 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:56:48.798 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:56:50.513 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:56:50.514 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:56:50.514 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:56:50.514 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:56:50.514 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:56:51.141 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T22:56:51.370 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:56:51.392 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:56:51.416 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:56:51.440 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:56:51.466 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:56:51.492 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:56:51.519 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:56:51.544 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:56:51.572 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:56:51.599 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:56:51.625 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:56:51.627 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:56:51.627 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:56:51.627 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:56:51.628 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:56:51.628 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:56:51.628 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:56:51.628 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:56:51.628 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:56:51.628 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:56:51.628 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:56:51.628 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:56:51.629 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:56:51.629 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:56:51.663 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:56:52.136 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:56:52.136 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:56:53.409 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:56:54.047 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T22:56:54.138 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:56:54.843 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:56:56.566 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:56:56.600 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:56:56.628 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:56:56.653 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:56:56.678 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:56:56.704 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:56:56.735 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:56:56.756 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:56:56.783 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:56:56.796 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:56:56.796 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:56:56.796 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:56:56.796 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:56:56.809 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:56:56.834 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:56:56.836 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:56:56.836 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:56:56.836 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:56:56.836 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:56:56.836 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:56:56.837 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:56:56.837 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:56:56.837 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:56:56.837 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:56:56.837 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:56:56.838 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:56:56.838 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:56:56.838 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:56:57.392 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T22:56:57.547 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:56:57.547 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:56:57.879 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:56:59.549 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:56:59.574 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:56:59.574 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:56:59.574 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:56:59.575 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:56:59.575 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:57:00.243 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T22:57:00.615 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:57:02.446 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:57:02.479 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:57:02.503 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:57:02.527 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:57:02.553 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:57:02.587 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:57:02.619 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:57:02.646 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:57:02.683 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:57:02.713 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:57:02.715 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:57:02.745 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:57:02.746 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:57:02.747 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:57:02.747 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:57:02.747 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:57:02.747 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:57:02.747 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:57:02.747 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:57:02.748 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:57:02.748 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:57:02.748 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:57:02.748 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:57:02.748 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:57:02.748 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:57:03.286 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T22:57:03.422 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:57:03.476 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:57:03.477 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:57:05.450 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:57:05.478 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:57:05.980 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T22:57:06.794 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:57:08.293 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:57:08.323 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:57:08.346 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:57:08.372 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:57:08.399 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:57:08.424 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:57:08.450 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:57:08.475 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:57:08.491 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:57:08.491 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:57:08.492 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:57:08.492 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:57:08.492 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:57:08.502 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:57:08.530 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:57:08.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:57:08.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:57:08.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:57:08.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:57:08.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:57:08.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:57:08.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:57:08.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:57:08.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:57:08.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:57:08.561 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:57:08.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:57:08.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:57:08.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:57:09.042 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T22:57:09.213 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:57:09.213 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:57:09.578 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:57:11.215 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:57:11.250 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:57:11.251 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:57:11.251 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:57:11.251 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:57:11.251 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:57:11.829 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T22:57:13.128 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:57:14.135 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:57:14.159 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:57:14.199 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:57:14.230 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:57:14.256 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:57:14.283 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:57:14.327 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:57:14.371 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:57:14.407 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:57:14.423 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:57:14.440 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:57:14.472 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:57:14.473 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:57:14.473 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:57:14.474 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:57:14.474 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:57:14.474 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:57:14.474 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:57:14.475 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:57:14.475 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:57:14.475 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:57:14.475 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:57:14.475 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:57:14.476 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:57:14.476 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:57:15.042 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:57:15.043 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T22:57:15.196 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:57:15.196 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:57:15.964 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:57:17.196 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:57:17.319 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:57:18.058 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:57:18.058 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T22:57:18.812 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:57:20.302 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:57:20.329 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:57:20.353 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:57:20.381 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:57:20.407 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:57:20.433 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:57:20.460 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:57:20.486 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:57:20.512 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:57:20.539 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:57:20.540 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:57:20.573 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:57:20.575 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:57:20.575 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:57:20.575 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:57:20.575 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:57:20.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:57:20.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:57:20.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:57:20.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:57:20.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:57:20.576 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:57:20.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:57:20.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:57:20.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:57:21.230 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:57:21.230 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T22:57:21.383 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:57:21.383 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:57:21.580 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:57:23.384 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:57:23.649 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:57:24.290 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:57:24.290 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T22:57:24.350 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:57:26.105 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:57:26.105 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:57:26.106 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:57:26.106 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:57:26.116 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:57:26.126 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:57:26.171 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:57:26.214 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:57:26.246 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:57:26.273 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:57:26.299 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:57:26.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:57:26.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:57:26.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:57:26.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:57:26.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:57:26.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:57:26.303 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:57:26.303 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:57:26.303 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:57:26.303 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:57:26.303 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:57:26.303 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:57:26.303 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:57:26.923 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:57:26.923 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:57:26.978 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:57:27.014 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:57:27.463 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:57:27.464 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T22:57:28.924 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:57:29.588 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:57:30.080 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:57:30.124 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:57:30.124 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T22:57:31.696 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:57:31.733 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:57:31.767 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:57:31.801 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:57:31.850 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:57:31.888 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:57:31.923 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:57:31.950 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:57:31.977 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:57:32.005 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:57:32.032 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:57:32.034 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:57:32.034 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:57:32.034 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:57:32.034 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:57:32.034 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:57:32.034 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:57:32.035 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:57:32.035 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:57:32.035 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:57:32.035 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:57:32.035 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:57:32.035 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:57:32.035 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:57:32.504 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:57:32.552 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:57:32.552 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:57:32.945 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:57:32.946 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T22:57:32.988 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:57:34.553 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:57:35.872 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:57:36.350 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:57:36.605 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:57:36.605 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T22:57:37.416 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:57:37.441 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:57:37.467 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:57:37.505 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:57:37.547 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:57:37.570 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:57:37.600 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:57:37.628 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:57:37.653 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:57:37.681 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:57:37.710 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:57:37.712 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:57:37.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:57:37.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:57:37.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:57:37.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:57:37.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:57:37.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:57:37.714 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:57:37.714 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:57:37.714 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:57:37.714 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:57:37.714 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:57:37.714 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:57:38.413 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:57:38.421 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:57:38.950 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:57:39.154 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:57:39.449 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:57:39.449 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T22:57:40.422 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:57:41.848 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:57:42.073 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:57:42.628 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:57:42.628 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T22:57:43.488 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:57:43.516 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:57:43.541 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:57:43.564 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:57:43.591 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:57:43.618 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:57:43.645 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:57:43.672 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:57:43.699 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:57:43.730 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:57:43.755 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:57:43.756 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:57:43.757 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:57:43.757 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:57:43.757 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:57:43.757 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:57:43.757 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:57:43.757 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:57:43.757 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:57:43.758 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:57:43.758 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:57:43.758 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:57:43.758 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:57:43.758 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:57:44.552 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:57:44.552 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:57:44.886 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:57:44.987 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:57:45.506 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:57:45.506 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T22:57:46.553 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:57:47.828 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:57:47.951 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:57:49.069 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:57:49.069 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T22:57:49.482 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:57:49.503 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:57:49.533 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:57:49.562 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:57:49.590 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:57:49.629 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:57:49.669 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:57:49.705 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:57:49.734 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:57:49.763 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:57:49.789 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:57:49.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:57:49.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:57:49.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:57:49.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:57:49.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:57:49.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:57:49.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:57:49.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:57:49.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:57:49.792 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:57:49.793 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:57:49.793 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:57:49.793 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:57:50.480 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:57:50.481 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:57:50.815 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:57:51.190 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:57:51.933 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:57:51.933 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T22:57:52.481 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:57:54.434 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:57:55.068 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:57:55.069 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T22:57:55.362 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:57:55.384 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:57:55.406 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:57:55.432 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:57:55.458 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:57:55.484 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:57:55.509 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:57:55.536 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:57:55.584 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:57:55.624 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:57:55.660 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:57:55.661 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:57:55.662 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:57:55.662 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:57:55.662 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:57:55.662 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:57:55.662 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:57:55.662 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:57:55.662 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:57:55.663 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:57:55.663 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:57:55.663 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:57:55.663 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:57:55.663 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:57:56.301 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:57:56.302 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:57:57.097 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:57:57.589 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:57:57.589 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T22:57:58.303 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:57:59.981 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:58:00.552 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:58:00.553 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T22:58:00.854 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:58:00.877 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:58:00.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:58:00.925 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:58:00.951 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:58:00.978 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:58:01.005 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:58:01.031 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:58:01.058 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:58:01.087 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:58:01.113 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:58:01.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:58:01.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:58:01.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:58:01.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:58:01.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:58:01.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:58:01.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:58:01.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:58:01.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:58:01.116 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:58:01.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:58:01.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:58:01.117 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:58:01.693 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:58:01.693 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:58:02.711 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:58:03.180 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:58:03.180 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T22:58:03.694 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:58:04.359 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:58:05.859 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:58:06.393 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:58:06.393 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T22:58:06.487 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:58:07.043 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:58:07.350 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:58:07.371 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:58:07.397 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:58:07.425 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:58:07.453 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:58:07.506 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:58:07.550 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:58:07.600 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:58:07.633 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:58:07.666 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:58:07.668 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:58:07.668 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:58:07.668 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:58:07.668 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:58:07.669 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:58:07.669 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:58:07.669 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:58:07.669 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:58:07.669 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:58:07.671 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:58:07.671 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:58:07.671 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:58:07.671 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:58:08.607 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:58:08.608 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:58:09.039 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:58:09.671 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:58:09.672 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T22:58:09.969 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:58:10.608 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:58:12.012 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:58:12.517 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:58:12.518 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T22:58:12.716 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:58:13.025 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:58:13.048 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:58:13.075 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:58:13.099 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:58:13.131 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:58:13.160 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:58:13.187 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:58:13.219 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:58:13.249 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:58:13.290 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:58:13.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:58:13.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:58:13.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:58:13.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:58:13.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:58:13.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:58:13.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:58:13.335 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:58:13.335 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:58:13.335 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:58:13.335 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:58:13.335 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:58:13.335 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:58:13.336 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:58:14.130 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:58:14.130 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:58:15.023 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:58:15.523 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:58:15.523 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T22:58:16.131 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:58:16.353 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:58:18.132 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:58:18.798 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T22:58:18.877 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:58:18.906 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:58:18.931 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:58:18.954 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:58:18.981 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:58:19.007 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:58:19.034 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:58:19.061 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:58:19.087 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:58:19.114 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:58:19.140 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:58:19.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:58:19.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:58:19.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:58:19.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:58:19.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:58:19.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:58:19.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:58:19.143 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:58:19.143 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:58:19.143 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:58:19.143 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:58:19.143 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:58:19.143 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:58:19.313 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:58:19.844 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:58:19.844 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:58:20.968 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:58:20.968 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:58:20.968 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:58:20.968 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:58:20.968 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:58:21.739 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T22:58:21.845 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:58:23.088 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:58:24.526 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:58:24.564 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:58:24.573 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:58:24.597 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:58:24.617 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:58:24.623 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:58:24.649 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:58:24.676 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:58:24.703 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:58:24.729 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:58:24.767 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:58:24.797 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:58:24.799 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:58:24.799 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:58:24.799 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:58:24.799 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:58:24.799 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:58:24.800 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:58:24.800 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:58:24.800 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:58:24.800 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:58:24.800 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:58:24.800 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:58:24.800 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:58:24.801 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:58:25.113 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T22:58:25.471 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:58:25.471 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:58:25.987 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:58:27.392 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:58:27.473 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:58:28.079 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T22:58:28.857 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:58:30.352 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:58:30.377 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:58:30.402 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:58:30.428 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:58:30.458 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:58:30.488 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:58:30.516 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:58:30.542 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:58:30.569 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:58:30.597 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:58:30.612 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:58:30.613 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:58:30.613 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T22:58:30.613 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T22:58:30.613 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:58:30.623 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:58:30.636 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:58:30.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:58:30.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:58:30.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:58:30.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:58:30.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:58:30.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:58:30.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:58:30.638 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:58:30.638 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:58:30.638 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:58:30.638 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:58:30.638 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:58:31.221 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T22:58:31.393 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:58:31.393 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:58:31.652 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:58:33.394 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:58:33.656 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:58:33.656 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:58:33.656 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:58:33.657 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:58:33.657 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:58:34.378 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T22:58:34.910 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:58:36.424 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:58:36.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:58:36.471 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:58:36.504 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:58:36.532 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:58:36.565 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:58:36.594 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:58:36.621 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:58:36.650 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:58:36.676 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:58:36.717 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:58:36.717 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:58:36.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:58:36.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:58:36.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:58:36.719 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:58:36.719 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:58:36.719 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:58:36.719 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:58:36.720 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:58:36.720 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:58:36.720 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:58:36.720 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:58:36.721 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:58:36.721 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:58:37.361 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:58:37.361 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:58:37.518 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T22:58:37.700 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:58:39.362 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:58:39.768 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:58:40.426 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:58:40.439 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T22:58:42.060 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:58:42.084 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:58:42.107 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:58:42.131 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:58:42.174 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:58:42.212 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:58:42.248 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:58:42.283 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:58:42.315 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:58:42.347 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:58:42.377 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:58:42.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:58:42.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:58:42.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:58:42.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:58:42.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:58:42.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:58:42.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:58:42.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:58:42.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:58:42.380 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:58:42.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:58:42.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:58:42.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:58:42.998 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:58:42.999 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:58:43.231 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:58:43.231 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T22:58:43.232 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T22:58:43.232 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T22:58:43.232 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:58:43.459 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:58:43.739 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T22:58:45.000 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:58:46.178 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:58:46.178 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:58:46.179 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:58:46.179 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:58:46.367 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:58:47.080 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T22:58:47.788 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:58:48.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:58:48.694 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:58:48.734 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:58:48.769 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:58:48.798 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:58:48.827 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:58:48.855 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:58:48.882 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:58:48.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:58:48.943 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:58:48.947 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:58:48.947 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:58:48.947 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:58:48.947 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:58:48.948 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:58:48.948 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:58:48.948 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:58:48.948 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:58:48.948 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:58:48.948 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:58:48.948 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:58:48.949 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:58:48.949 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:58:49.053 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:58:49.311 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:58:49.555 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:58:49.556 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:58:49.740 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T22:58:51.557 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:58:51.898 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T22:58:52.382 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:58:52.465 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T22:58:54.435 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:58:54.471 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:58:54.501 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:58:54.528 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:58:54.558 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:58:54.585 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:58:54.612 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:58:54.664 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:58:54.708 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:58:54.743 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:58:54.772 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:58:54.774 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:58:54.774 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:58:54.774 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:58:54.774 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:58:54.774 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:58:54.774 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:58:54.774 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:58:54.775 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:58:54.775 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:58:54.775 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:58:54.775 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:58:54.775 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:58:54.775 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:58:54.973 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:58:55.025 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:58:55.359 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:58:55.359 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:58:55.898 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:58:55.899 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T22:58:57.361 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:58:58.405 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:58:58.490 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:58:59.363 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:58:59.364 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T22:58:59.918 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:59:00.773 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:59:00.815 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:59:00.850 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:59:00.883 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:59:00.910 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:59:00.937 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:59:00.964 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:59:00.992 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:59:01.019 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:59:01.046 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:59:01.048 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:59:01.048 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:59:01.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:59:01.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:59:01.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:59:01.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:59:01.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:59:01.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:59:01.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:59:01.050 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:59:01.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:59:01.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:59:01.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:59:01.362 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:59:01.497 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:59:01.541 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:59:01.541 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:59:01.998 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:59:01.998 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T22:59:03.542 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:59:04.203 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:59:04.746 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:59:04.746 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T22:59:04.871 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:59:06.385 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:59:07.237 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:59:07.288 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:59:07.307 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:59:07.327 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:59:07.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:59:07.399 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:59:07.435 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:59:07.464 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:59:07.492 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:59:07.526 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:59:07.556 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:59:07.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:59:07.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:59:07.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:59:07.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:59:07.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:59:07.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:59:07.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:59:07.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:59:07.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:59:07.561 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:59:07.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:59:07.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:59:07.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:59:07.675 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:59:07.830 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:59:07.831 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T22:59:08.139 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:59:08.139 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:59:10.140 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:59:10.263 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:59:10.891 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:59:10.892 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T22:59:11.101 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:59:13.089 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:59:13.612 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:59:13.950 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:59:13.998 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:59:14.037 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:59:14.082 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:59:14.114 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:59:14.114 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T22:59:14.138 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:59:14.160 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:59:14.177 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:59:14.206 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:59:14.233 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:59:14.259 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:59:14.298 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:59:14.299 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:59:14.300 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:59:14.300 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:59:14.300 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:59:14.300 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:59:14.300 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:59:14.300 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:59:14.300 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:59:14.301 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:59:14.301 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:59:14.301 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:59:14.301 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:59:14.301 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:59:15.253 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:59:15.254 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:59:16.622 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:59:17.133 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:59:17.134 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T22:59:17.255 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:59:17.878 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:59:19.612 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:59:20.155 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:59:20.169 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:59:20.169 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T22:59:20.188 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:59:20.217 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:59:20.242 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:59:20.269 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:59:20.297 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:59:20.322 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:59:20.348 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:59:20.375 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:59:20.403 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:59:20.429 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:59:20.430 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:59:20.430 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:59:20.430 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:59:20.430 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:59:20.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:59:20.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:59:20.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:59:20.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:59:20.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:59:20.431 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:59:20.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:59:20.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:59:20.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:59:20.851 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:59:20.971 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:59:20.971 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:59:22.431 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:59:22.972 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:59:23.045 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:59:23.045 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T22:59:23.897 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:59:25.776 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:59:25.810 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:59:25.842 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:59:25.872 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:59:25.900 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:59:25.927 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:59:25.953 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:59:25.979 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:59:26.002 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:59:26.014 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:59:26.041 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:59:26.067 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:59:26.075 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:59:26.075 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:59:26.075 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:59:26.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:59:26.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:59:26.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:59:26.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:59:26.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:59:26.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:59:26.076 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:59:26.077 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:59:26.077 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:59:26.077 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:59:26.647 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:59:26.647 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T22:59:26.691 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:59:26.802 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:59:26.803 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:59:28.803 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:59:29.023 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:59:29.179 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:59:29.704 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:59:29.704 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T22:59:31.386 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:59:31.414 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:59:31.438 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:59:31.461 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:59:31.488 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:59:31.514 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:59:31.540 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:59:31.565 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:59:31.592 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:59:31.618 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:59:31.644 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:59:31.646 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:59:31.646 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:59:31.646 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:59:31.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:59:31.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:59:31.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:59:31.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:59:31.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:59:31.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:59:31.647 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:59:31.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:59:31.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:59:31.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:59:32.154 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:59:32.155 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:59:32.208 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:59:32.230 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:59:32.706 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:59:32.707 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T22:59:34.156 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:59:34.689 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:59:35.239 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:59:35.308 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:59:35.308 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T22:59:36.741 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:59:36.763 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:59:36.785 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:59:36.808 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:59:36.834 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:59:36.861 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:59:36.888 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:59:36.914 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:59:36.940 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:59:36.967 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:59:36.994 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:59:36.996 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:59:36.996 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:59:36.996 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:59:36.996 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:59:36.996 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:59:36.996 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:59:36.997 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:59:36.997 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:59:36.997 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:59:36.997 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:59:36.997 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:59:36.997 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:59:36.997 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:59:37.618 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:59:37.618 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:59:37.859 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:59:38.087 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T22:59:38.367 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:59:38.367 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T22:59:39.619 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:59:40.833 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T22:59:40.958 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:59:41.941 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:59:41.941 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T22:59:42.490 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:59:42.525 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:59:42.556 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:59:42.586 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:59:42.615 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:59:42.643 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:59:42.669 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:59:42.695 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:59:42.722 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:59:42.752 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:59:42.785 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:59:42.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:59:42.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:59:42.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:59:42.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:59:42.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:59:42.788 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:59:42.788 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:59:42.788 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:59:42.788 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:59:42.789 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:59:42.789 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:59:42.789 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:59:42.789 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:59:43.467 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T22:59:43.552 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:59:43.553 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:59:44.135 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:59:44.690 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:59:44.690 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T22:59:45.554 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:59:46.386 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T22:59:47.436 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:59:48.125 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:59:48.126 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T22:59:48.391 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:59:48.414 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:59:48.437 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:59:48.460 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:59:48.487 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:59:48.513 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:59:48.540 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:59:48.566 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:59:48.603 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:59:48.620 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:59:48.646 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:59:48.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:59:48.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:59:48.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:59:48.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:59:48.650 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:59:48.650 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:59:48.650 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:59:48.650 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:59:48.650 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:59:48.650 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:59:48.651 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:59:48.651 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:59:48.651 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T22:59:49.089 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T22:59:49.125 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:59:49.125 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:59:50.410 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:59:51.022 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:59:51.023 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T22:59:51.126 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T22:59:51.889 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T22:59:53.476 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:59:54.030 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:59:54.030 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T22:59:54.114 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:59:54.135 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:59:54.162 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:59:54.185 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:59:54.211 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:59:54.237 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:59:54.264 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:59:54.302 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:59:54.327 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:59:54.354 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:59:54.389 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:59:54.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T22:59:54.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T22:59:54.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T22:59:54.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T22:59:54.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T22:59:54.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T22:59:54.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T22:59:54.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T22:59:54.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T22:59:54.391 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:59:54.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:59:54.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T22:59:54.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T22:59:54.821 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T22:59:55.012 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T22:59:55.012 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T22:59:56.309 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T22:59:56.902 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T22:59:56.902 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T22:59:57.014 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T22:59:57.913 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T22:59:59.509 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T22:59:59.509 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T22:59:59.509 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T22:59:59.510 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T22:59:59.668 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T22:59:59.694 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T22:59:59.713 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T22:59:59.736 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T22:59:59.763 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T22:59:59.789 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T22:59:59.815 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T22:59:59.841 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T22:59:59.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T22:59:59.933 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T22:59:59.971 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T22:59:59.973 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T22:59:59.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T22:59:59.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T22:59:59.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T22:59:59.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T22:59:59.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T22:59:59.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T22:59:59.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T22:59:59.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T22:59:59.975 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T22:59:59.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T22:59:59.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T22:59:59.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:00:00.051 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:00:00.465 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:00:00.465 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:00:00.865 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:00:02.448 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:00:02.466 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:00:03.211 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:00:04.386 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:00:05.405 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:00:05.765 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:00:05.765 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:00:05.765 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:00:05.765 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:00:05.766 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:00:06.218 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:00:06.268 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:00:06.311 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:00:06.345 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:00:06.378 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:00:06.414 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:00:06.437 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:00:06.463 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:00:06.490 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:00:06.518 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:00:06.544 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:00:06.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:00:06.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:00:06.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:00:06.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:00:06.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:00:06.546 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:00:06.546 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:00:06.546 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:00:06.546 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:00:06.546 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:00:06.546 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:00:06.546 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:00:06.547 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:00:07.120 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:00:07.151 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:00:07.151 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:00:08.405 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:00:08.406 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:00:08.406 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:00:08.406 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:00:09.138 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:00:09.153 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:00:11.472 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:00:11.497 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:00:11.533 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:00:11.563 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:00:11.592 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:00:11.623 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:00:11.650 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:00:11.676 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:00:11.704 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:00:11.731 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:00:11.744 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:00:11.756 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:00:11.757 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:00:11.758 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:00:11.758 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:00:11.758 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:00:11.758 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:00:11.758 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:00:11.758 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:00:11.758 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:00:11.759 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:00:11.759 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:00:11.759 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:00:11.759 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:00:11.759 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:00:12.411 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:00:12.556 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:00:12.556 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:00:14.415 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:00:14.559 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:00:14.958 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:00:16.978 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:00:17.239 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:00:17.788 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:00:17.829 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:00:17.889 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:00:17.918 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:00:17.952 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:00:17.979 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:00:18.007 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:00:18.034 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:00:18.062 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:00:18.089 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:00:18.116 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:00:18.117 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:00:18.117 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:00:18.117 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:00:18.118 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:00:18.118 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:00:18.118 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:00:18.118 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:00:18.118 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:00:18.118 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:00:18.118 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:00:18.119 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:00:18.119 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:00:18.119 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:00:18.695 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:00:18.695 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:00:20.047 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:00:20.047 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:00:20.047 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:00:20.047 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:00:20.047 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:00:20.389 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:00:20.567 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:00:20.697 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:00:22.928 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:00:23.035 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:00:23.053 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:00:23.077 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:00:23.100 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:00:23.136 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:00:23.155 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:00:23.161 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:00:23.190 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:00:23.216 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:00:23.243 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:00:23.273 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:00:23.301 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:00:23.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:00:23.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:00:23.303 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:00:23.303 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:00:23.303 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:00:23.303 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:00:23.303 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:00:23.303 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:00:23.303 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:00:23.304 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:00:23.304 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:00:23.304 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:00:23.304 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:00:23.995 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:00:24.201 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:00:24.201 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:00:26.202 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:00:26.221 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:00:26.387 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:00:26.722 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:00:28.911 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:00:28.938 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:00:28.964 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:00:28.993 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:00:29.017 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:00:29.051 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:00:29.081 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:00:29.109 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:00:29.135 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:00:29.144 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:00:29.144 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:00:29.144 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:00:29.144 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:00:29.145 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:00:29.163 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:00:29.188 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:00:29.189 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:00:29.189 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:00:29.189 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:00:29.190 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:00:29.190 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:00:29.190 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:00:29.190 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:00:29.190 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:00:29.190 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:00:29.191 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:00:29.191 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:00:29.191 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:00:29.191 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:00:29.248 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:00:29.907 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:00:29.907 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:00:29.907 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:00:31.908 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:00:32.019 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:00:32.019 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:00:32.020 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:00:32.020 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:00:32.020 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:00:32.526 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:00:32.935 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:00:34.746 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:00:34.779 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:00:34.808 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:00:34.835 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:00:34.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:00:34.889 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:00:34.915 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:00:34.941 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:00:34.950 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:00:34.968 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:00:34.994 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:00:35.020 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:00:35.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:00:35.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:00:35.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:00:35.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:00:35.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:00:35.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:00:35.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:00:35.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:00:35.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:00:35.023 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:00:35.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:00:35.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:00:35.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:00:35.569 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:00:35.569 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:00:35.657 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:00:35.716 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:00:35.717 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:00:37.719 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:00:37.830 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:00:38.452 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:00:38.453 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:00:38.926 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:00:40.595 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:00:40.620 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:00:40.647 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:00:40.668 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:00:40.694 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:00:40.719 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:00:40.720 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:00:40.747 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:00:40.773 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:00:40.799 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:00:40.826 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:00:40.851 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:00:40.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:00:40.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:00:40.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:00:40.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:00:40.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:00:40.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:00:40.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:00:40.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:00:40.855 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:00:40.855 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:00:40.855 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:00:40.855 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:00:40.855 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:00:41.201 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:00:41.202 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:00:41.568 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:00:41.568 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:00:41.884 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:00:43.432 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:00:43.570 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:00:44.170 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:00:44.172 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:00:44.569 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:00:46.394 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:00:46.420 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:00:46.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:00:46.471 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:00:46.499 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:00:46.525 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:00:46.572 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:00:46.619 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:00:46.660 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:00:46.669 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:00:46.694 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:00:46.741 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:00:46.744 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:00:46.744 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:00:46.744 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:00:46.744 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:00:46.744 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:00:46.744 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:00:46.745 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:00:46.745 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:00:46.745 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:00:46.745 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:00:46.745 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:00:46.745 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:00:46.745 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:00:47.287 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:00:47.288 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:00:47.365 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:00:47.436 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:00:47.436 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:00:49.437 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:00:49.568 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:00:50.299 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:00:50.300 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:00:50.391 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:00:52.556 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:00:52.587 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:00:52.607 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:00:52.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:00:52.641 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:00:52.668 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:00:52.694 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:00:52.721 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:00:52.761 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:00:52.796 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:00:52.831 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:00:52.866 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:00:52.868 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:00:52.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:00:52.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:00:52.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:00:52.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:00:52.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:00:52.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:00:52.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:00:52.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:00:52.870 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:00:52.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:00:52.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:00:52.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:00:53.065 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:00:53.065 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:00:53.238 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:00:53.400 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:00:53.400 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:00:55.272 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:00:55.402 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:00:55.843 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:00:55.844 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:00:56.426 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:00:58.153 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:00:58.185 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:00:58.215 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:00:58.244 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:00:58.271 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:00:58.298 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:00:58.326 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:00:58.350 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:00:58.358 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:00:58.378 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:00:58.403 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:00:58.430 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:00:58.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:00:58.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:00:58.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:00:58.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:00:58.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:00:58.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:00:58.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:00:58.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:00:58.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:00:58.434 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:00:58.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:00:58.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:00:58.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:00:58.967 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:00:58.967 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:00:59.109 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:00:59.109 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:00:59.381 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:01:01.110 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:01:01.308 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:01:01.936 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:01:01.976 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:01:01.977 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:01:03.903 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:01:03.926 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:01:03.950 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:01:03.976 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:01:04.004 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:01:04.030 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:01:04.059 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:01:04.110 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:01:04.150 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:01:04.185 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:01:04.215 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:01:04.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:01:04.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:01:04.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:01:04.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:01:04.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:01:04.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:01:04.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:01:04.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:01:04.220 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:01:04.220 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:01:04.220 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:01:04.220 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:01:04.220 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:01:04.447 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:01:04.523 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:01:04.765 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:01:04.766 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:01:05.097 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:01:05.097 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:01:06.768 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:01:07.348 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:01:07.910 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:01:07.911 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:01:09.263 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:01:09.304 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:01:09.336 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:01:09.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:01:09.401 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:01:09.422 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:01:09.448 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:01:09.475 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:01:09.501 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:01:09.532 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:01:09.556 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:01:09.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:01:09.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:01:09.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:01:09.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:01:09.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:01:09.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:01:09.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:01:09.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:01:09.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:01:09.560 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:01:09.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:01:09.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:01:09.566 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:01:10.089 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:01:10.090 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:01:10.094 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:01:10.569 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:01:10.570 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:01:12.091 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:01:12.565 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:01:13.374 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:01:13.374 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:01:14.801 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:01:14.831 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:01:14.853 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:01:14.872 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:01:14.898 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:01:14.924 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:01:14.951 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:01:14.978 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:01:15.005 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:01:15.032 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:01:15.059 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:01:15.063 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:01:15.063 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:01:15.063 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:01:15.063 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:01:15.063 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:01:15.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:01:15.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:01:15.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:01:15.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:01:15.064 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:01:15.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:01:15.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:01:15.065 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:01:15.244 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:01:15.577 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:01:15.578 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:01:15.962 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:01:15.963 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:01:17.578 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:01:17.823 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:01:17.952 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:01:18.518 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:01:18.519 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:01:20.246 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:01:20.276 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:01:20.299 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:01:20.323 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:01:20.349 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:01:20.375 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:01:20.402 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:01:20.427 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:01:20.455 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:01:20.482 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:01:20.511 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:01:20.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:01:20.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:01:20.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:01:20.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:01:20.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:01:20.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:01:20.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:01:20.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:01:20.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:01:20.513 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:01:20.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:01:20.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:01:20.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:01:20.797 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:01:20.941 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:01:20.999 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:01:20.999 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:01:21.365 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:01:21.365 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:01:23.000 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:01:23.589 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:01:24.006 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:01:24.185 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:01:24.186 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:01:25.743 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:01:25.767 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:01:25.791 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:01:25.816 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:01:25.842 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:01:25.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:01:25.897 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:01:25.943 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:01:25.979 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:01:26.019 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:01:26.059 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:01:26.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:01:26.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:01:26.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:01:26.065 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:01:26.065 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:01:26.065 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:01:26.065 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:01:26.065 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:01:26.065 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:01:26.065 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:01:26.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:01:26.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:01:26.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:01:26.670 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:01:26.670 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:01:26.765 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:01:26.920 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:01:27.576 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:01:27.576 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:01:28.672 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:01:29.322 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:01:30.479 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:01:31.117 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:01:31.118 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:01:31.460 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:01:31.482 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:01:31.506 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:01:31.529 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:01:31.555 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:01:31.582 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:01:31.608 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:01:31.634 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:01:31.661 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:01:31.688 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:01:31.715 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:01:31.716 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:01:31.717 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:01:31.717 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:01:31.717 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:01:31.717 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:01:31.717 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:01:31.717 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:01:31.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:01:31.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:01:31.718 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:01:31.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:01:31.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:01:31.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:01:32.128 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:01:32.248 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:01:32.248 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:01:33.298 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:01:33.968 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:01:33.968 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:01:34.249 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:01:35.387 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:01:36.540 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:01:37.043 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:01:37.044 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:01:37.071 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:01:37.098 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:01:37.123 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:01:37.150 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:01:37.176 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:01:37.202 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:01:37.229 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:01:37.257 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:01:37.291 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:01:37.324 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:01:37.326 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:01:37.326 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:01:37.326 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:01:37.326 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:01:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:01:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:01:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:01:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:01:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:01:37.328 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:01:37.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:01:37.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:01:37.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:01:37.976 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:01:37.976 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:01:38.108 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:01:39.005 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:01:39.005 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:01:39.005 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:01:39.005 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:01:39.005 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:01:39.887 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:01:39.977 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:01:40.691 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:01:42.570 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:01:42.571 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:01:42.571 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:01:42.571 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:01:42.571 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:01:42.792 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:01:43.139 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:01:43.648 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:01:43.692 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:01:43.728 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:01:43.764 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:01:43.797 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:01:43.813 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:01:43.826 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:01:43.855 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:01:43.883 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:01:43.912 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:01:43.938 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:01:43.941 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:01:43.941 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:01:43.941 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:01:43.941 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:01:43.941 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:01:43.941 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:01:43.941 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:01:43.942 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:01:43.942 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:01:43.942 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:01:43.942 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:01:43.942 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:01:43.942 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:01:44.862 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:01:44.862 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:01:45.552 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:01:46.195 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:01:46.863 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:01:46.964 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:01:49.023 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:01:49.638 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:01:49.674 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:01:49.709 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:01:49.710 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:01:49.733 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:01:49.756 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:01:49.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:01:49.804 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:01:49.832 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:01:49.861 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:01:49.906 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:01:49.929 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:01:49.955 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:01:49.956 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:01:49.957 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:01:49.957 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:01:49.957 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:01:49.957 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:01:49.957 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:01:49.957 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:01:49.958 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:01:49.958 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:01:49.958 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:01:49.958 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:01:49.958 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:01:49.958 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:01:50.956 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:01:50.956 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:01:52.609 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:01:52.609 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:01:52.609 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:01:52.610 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:01:52.610 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:01:52.958 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:01:52.966 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:01:53.102 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:01:55.476 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:01:55.500 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:01:55.742 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:01:55.742 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:01:55.742 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:01:55.742 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:01:56.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:01:56.391 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:01:56.392 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:01:56.427 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:01:56.463 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:01:56.491 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:01:56.517 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:01:56.545 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:01:56.571 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:01:56.601 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:01:56.628 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:01:56.629 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:01:56.630 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:01:56.630 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:01:56.630 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:01:56.630 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:01:56.630 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:01:56.630 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:01:56.630 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:01:56.631 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:01:56.631 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:01:56.631 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:01:56.631 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:01:56.631 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:01:57.487 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:01:57.488 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:01:58.996 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:01:59.001 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:01:59.489 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:01:59.816 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:02:01.658 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:02:01.893 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:02:02.067 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:02:02.650 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:02:02.747 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:02:02.794 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:02:02.832 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:02:02.866 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:02:02.900 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:02:02.922 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:02:02.948 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:02:02.974 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:02:03.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:02:03.028 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:02:03.030 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:02:03.031 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:02:03.031 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:02:03.031 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:02:03.031 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:02:03.031 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:02:03.031 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:02:03.031 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:02:03.032 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:02:03.032 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:02:03.032 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:02:03.032 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:02:03.032 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:02:03.703 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:02:03.703 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:02:05.087 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:02:05.170 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:02:05.170 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:02:05.170 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:02:05.170 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:02:05.704 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:02:05.889 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:02:07.728 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:02:07.928 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:02:08.246 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:02:08.247 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:02:08.247 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:02:08.247 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:02:08.247 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:02:08.783 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:02:08.809 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:02:08.822 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:02:08.853 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:02:08.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:02:08.924 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:02:08.955 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:02:08.983 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:02:09.013 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:02:09.040 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:02:09.064 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:02:09.065 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:02:09.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:02:09.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:02:09.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:02:09.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:02:09.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:02:09.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:02:09.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:02:09.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:02:09.067 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:02:09.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:02:09.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:02:09.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:02:09.732 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:02:09.732 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:02:10.981 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:02:11.155 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:02:11.735 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:02:11.828 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:02:13.673 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:02:13.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:02:13.877 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:02:13.903 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:02:13.930 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:02:13.961 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:02:13.984 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:02:14.011 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:02:14.041 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:02:14.068 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:02:14.091 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:02:14.116 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:02:14.119 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:02:14.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:02:14.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:02:14.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:02:14.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:02:14.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:02:14.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:02:14.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:02:14.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:02:14.123 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:02:14.123 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:02:14.123 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:02:14.123 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:02:14.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:02:14.976 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:02:14.976 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:02:14.982 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:02:14.982 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:02:16.983 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:02:17.044 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:02:17.117 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:02:17.986 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:02:17.987 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:02:19.570 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:02:19.592 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:02:19.616 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:02:19.654 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:02:19.673 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:02:19.701 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:02:19.719 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:02:19.733 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:02:19.755 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:02:19.785 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:02:19.812 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:02:19.839 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:02:19.840 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:02:19.840 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:02:19.841 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:02:19.841 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:02:19.841 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:02:19.841 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:02:19.841 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:02:19.841 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:02:19.841 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:02:19.842 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:02:19.842 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:02:19.842 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:02:19.842 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:02:20.361 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:02:20.529 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:02:20.530 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:02:21.016 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:02:21.017 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:02:22.531 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:02:22.995 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:02:23.677 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:02:24.149 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:02:24.149 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:02:25.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:02:25.700 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:02:26.339 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:02:26.372 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:02:26.393 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:02:26.396 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:02:26.422 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:02:26.449 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:02:26.484 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:02:26.504 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:02:26.540 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:02:26.567 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:02:26.595 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:02:26.597 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:02:26.597 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:02:26.597 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:02:26.597 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:02:26.598 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:02:26.598 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:02:26.598 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:02:26.598 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:02:26.598 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:02:26.598 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:02:26.599 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:02:26.599 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:02:26.599 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:02:26.930 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:02:26.931 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:02:27.521 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:02:27.521 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:02:29.048 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:02:29.528 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:02:29.529 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:02:29.928 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:02:29.929 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:02:31.546 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:02:31.964 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:02:32.033 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:02:32.665 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:02:32.666 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:02:32.835 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:02:32.861 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:02:32.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:02:32.918 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:02:32.938 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:02:32.965 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:02:33.009 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:02:33.040 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:02:33.068 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:02:33.090 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:02:33.091 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:02:33.091 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:02:33.091 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:02:33.091 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:02:33.092 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:02:33.092 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:02:33.092 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:02:33.092 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:02:33.092 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:02:33.092 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:02:33.092 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:02:33.093 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:02:33.093 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:02:33.985 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:02:33.985 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:02:35.102 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:02:35.328 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:02:35.986 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:02:36.011 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:02:36.012 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:02:37.624 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:02:38.189 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:02:38.209 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:02:38.232 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:02:38.261 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:02:38.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:02:38.327 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:02:38.360 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:02:38.396 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:02:38.416 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:02:38.469 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:02:38.495 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:02:38.518 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:02:38.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:02:38.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:02:38.520 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:02:38.520 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:02:38.520 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:02:38.520 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:02:38.520 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:02:38.520 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:02:38.521 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:02:38.521 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:02:38.521 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:02:38.521 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:02:38.521 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:02:39.153 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:02:39.153 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:02:39.449 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:02:39.449 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:02:41.133 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:02:41.364 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:02:41.450 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:02:42.260 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:02:42.261 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:02:43.950 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:02:44.041 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:02:44.644 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:02:44.925 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:02:44.976 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:02:45.016 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:02:45.050 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:02:45.093 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:02:45.135 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:02:45.166 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:02:45.194 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:02:45.225 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:02:45.226 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:02:45.227 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:02:45.260 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:02:45.261 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:02:45.261 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:02:45.261 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:02:45.262 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:02:45.262 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:02:45.262 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:02:45.262 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:02:45.262 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:02:45.263 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:02:45.263 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:02:45.263 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:02:45.263 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:02:45.263 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:02:46.082 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:02:46.082 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:02:46.915 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:02:47.903 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:02:48.083 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:02:48.457 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:02:48.458 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:02:49.480 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:02:50.344 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:02:50.769 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:02:51.197 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:02:51.248 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:02:51.289 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:02:51.340 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:02:51.377 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:02:51.411 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:02:51.439 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:02:51.482 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:02:51.518 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:02:51.550 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:02:51.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:02:51.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:02:51.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:02:51.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:02:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:02:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:02:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:02:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:02:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:02:51.554 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:02:51.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:02:51.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:02:51.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:02:51.596 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:02:51.596 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:02:52.229 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:02:52.229 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:02:53.006 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:02:53.946 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:02:54.231 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:02:54.425 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:02:54.425 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:02:55.616 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:02:57.070 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:02:57.071 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:02:57.071 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:02:57.071 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:02:57.071 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:02:57.077 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:02:57.103 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:02:57.131 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:02:57.156 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:02:57.183 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:02:57.209 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:02:57.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:02:57.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:02:57.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:02:57.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:02:57.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:02:57.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:02:57.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:02:57.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:02:57.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:02:57.213 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:02:57.213 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:02:57.213 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:02:57.213 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:02:57.320 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:02:57.757 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:02:57.758 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:02:58.081 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:02:58.082 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:02:59.031 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:02:59.759 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:03:00.377 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:03:00.991 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:03:00.992 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:03:01.851 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:03:02.196 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:03:02.224 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:03:02.243 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:03:02.266 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:03:02.292 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:03:02.318 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:03:02.344 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:03:02.371 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:03:02.408 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:03:02.425 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:03:02.451 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:03:02.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:03:02.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:03:02.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:03:02.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:03:02.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:03:02.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:03:02.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:03:02.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:03:02.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:03:02.454 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:03:02.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:03:02.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:03:02.455 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:03:03.325 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:03:03.325 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:03:03.998 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:03:04.607 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:03:04.607 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:03:05.061 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:03:05.326 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:03:06.779 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:03:07.441 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:03:07.442 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:03:07.660 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:03:07.997 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:03:08.029 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:03:08.060 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:03:08.083 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:03:08.110 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:03:08.137 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:03:08.162 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:03:08.189 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:03:08.222 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:03:08.248 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:03:08.275 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:03:08.276 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:03:08.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:03:08.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:03:08.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:03:08.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:03:08.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:03:08.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:03:08.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:03:08.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:03:08.278 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:03:08.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:03:08.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:03:08.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:03:08.929 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:03:08.929 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:03:09.934 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:03:10.514 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:03:10.514 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:03:10.930 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:03:11.024 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:03:12.955 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:03:13.533 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:03:13.533 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:03:13.568 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:03:13.765 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:03:13.789 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:03:13.812 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:03:13.835 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:03:13.861 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:03:13.887 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:03:13.916 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:03:13.942 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:03:13.970 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:03:13.997 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:03:14.023 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:03:14.026 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:03:14.026 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:03:14.026 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:03:14.026 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:03:14.027 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:03:14.027 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:03:14.027 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:03:14.027 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:03:14.027 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:03:14.027 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:03:14.027 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:03:14.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:03:14.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:03:14.910 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:03:14.910 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:03:16.104 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:03:16.631 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:03:16.632 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:03:16.911 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:03:17.057 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:03:19.091 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:03:19.091 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:03:19.092 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:03:19.092 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:03:19.566 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:03:19.573 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:03:19.592 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:03:19.615 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:03:19.639 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:03:19.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:03:19.695 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:03:19.721 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:03:19.749 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:03:19.776 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:03:19.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:03:19.828 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:03:19.831 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:03:19.831 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:03:19.832 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:03:19.832 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:03:19.832 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:03:19.832 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:03:19.832 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:03:19.833 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:03:19.833 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:03:19.833 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:03:19.833 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:03:19.833 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:03:19.833 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:03:19.843 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:03:20.528 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:03:20.529 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:03:22.064 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:03:22.530 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:03:22.554 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:03:23.019 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:03:25.172 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:03:25.241 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:03:25.242 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:03:25.242 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:03:25.242 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:03:25.242 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:03:25.739 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:03:25.748 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:03:26.023 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:03:26.048 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:03:26.070 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:03:26.096 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:03:26.125 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:03:26.148 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:03:26.174 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:03:26.200 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:03:26.227 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:03:26.259 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:03:26.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:03:26.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:03:26.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:03:26.260 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:03:26.260 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:03:26.260 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:03:26.260 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:03:26.260 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:03:26.260 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:03:26.260 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:03:26.261 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:03:26.261 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:03:26.261 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:03:27.015 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:03:27.016 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:03:28.376 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:03:28.376 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:03:28.376 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:03:28.377 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:03:28.864 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:03:29.017 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:03:29.018 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:03:31.460 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:03:31.492 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:03:31.519 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:03:31.551 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:03:31.575 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:03:31.602 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:03:31.605 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:03:31.630 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:03:31.656 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:03:31.682 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:03:31.715 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:03:31.721 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:03:31.741 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:03:31.743 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:03:31.743 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:03:31.743 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:03:31.743 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:03:31.744 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:03:31.744 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:03:31.744 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:03:31.744 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:03:31.744 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:03:31.744 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:03:31.745 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:03:31.745 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:03:31.745 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:03:32.332 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:03:32.613 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:03:32.614 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:03:34.563 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:03:34.617 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:03:35.071 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:03:35.203 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:03:37.055 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:03:37.680 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:03:37.907 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:03:37.917 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:03:37.958 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:03:38.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:03:38.025 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:03:38.053 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:03:38.100 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:03:38.148 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:03:38.179 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:03:38.181 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:03:38.206 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:03:38.243 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:03:38.244 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:03:38.244 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:03:38.244 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:03:38.245 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:03:38.245 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:03:38.245 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:03:38.245 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:03:38.245 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:03:38.246 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:03:38.246 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:03:38.246 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:03:38.246 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:03:38.246 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:03:38.906 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:03:38.907 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:03:40.723 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:03:40.784 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:03:40.784 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:03:40.784 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:03:40.784 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:03:40.785 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:03:40.908 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:03:41.701 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:03:43.411 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:03:43.449 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:03:43.476 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:03:43.501 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:03:43.526 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:03:43.552 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:03:43.580 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:03:43.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:03:43.633 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:03:43.659 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:03:43.686 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:03:43.711 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:03:43.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:03:43.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:03:43.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:03:43.714 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:03:43.714 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:03:43.714 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:03:43.714 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:03:43.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:03:43.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:03:43.715 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:03:43.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:03:43.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:03:43.716 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:03:44.142 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:03:44.438 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:03:44.438 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:03:44.823 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:03:46.438 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:03:46.440 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:03:46.933 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:03:47.601 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:03:49.037 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:03:49.288 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:03:49.309 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:03:49.334 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:03:49.357 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:03:49.383 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:03:49.410 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:03:49.436 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:03:49.463 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:03:49.490 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:03:49.517 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:03:49.543 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:03:49.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:03:49.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:03:49.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:03:49.546 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:03:49.546 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:03:49.546 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:03:49.552 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:03:49.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:03:49.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:03:49.553 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:03:49.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:03:49.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:03:49.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:03:50.026 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:03:50.027 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:03:50.027 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:03:50.027 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:03:50.027 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:03:50.187 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:03:50.187 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:03:50.574 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:03:52.189 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:03:52.703 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:03:52.704 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:03:52.704 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:03:52.704 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:03:52.705 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:03:53.045 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:03:53.515 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:03:54.809 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:03:54.835 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:03:54.863 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:03:54.888 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:03:54.934 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:03:54.974 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:03:55.010 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:03:55.048 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:03:55.075 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:03:55.116 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:03:55.149 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:03:55.150 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:03:55.151 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:03:55.151 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:03:55.151 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:03:55.152 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:03:55.152 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:03:55.152 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:03:55.152 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:03:55.153 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:03:55.153 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:03:55.153 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:03:55.153 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:03:55.154 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:03:55.749 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:03:55.750 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:03:55.787 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:03:56.025 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:03:56.662 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:03:56.663 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:03:57.751 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:03:59.055 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:03:59.499 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:00.339 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:00.340 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:04:00.744 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:04:01.602 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:04:01.659 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:04:01.696 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:04:01.741 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:04:01.772 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:04:01.800 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:04:01.828 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:04:01.855 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:04:01.883 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:04:01.910 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:04:01.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:04:01.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:04:01.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:04:01.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:04:01.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:04:01.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:04:01.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:04:01.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:04:01.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:04:01.915 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:04:01.916 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:04:01.916 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:04:01.916 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:04:02.066 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:04:02.393 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:02.552 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:04:02.552 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:04:03.030 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:03.030 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:04:04.553 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:04:05.081 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:04:05.367 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:06.355 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:06.355 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:04:07.187 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:04:07.212 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:04:07.236 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:04:07.259 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:04:07.285 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:04:07.312 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:04:07.339 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:04:07.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:04:07.395 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:04:07.420 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:04:07.446 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:04:07.448 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:04:07.448 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:04:07.448 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:04:07.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:04:07.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:04:07.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:04:07.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:04:07.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:04:07.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:04:07.450 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:04:07.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:04:07.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:04:07.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:04:08.064 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:04:08.141 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:04:08.141 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:04:08.478 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:09.227 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:09.227 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:04:10.146 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:04:11.468 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:04:11.618 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:12.433 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:12.434 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:04:12.824 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:04:12.848 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:04:12.871 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:04:12.895 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:04:12.922 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:04:12.948 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:04:12.975 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:04:13.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:04:13.028 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:04:13.055 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:04:13.080 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:04:13.082 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:04:13.082 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:04:13.083 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:04:13.083 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:04:13.083 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:04:13.083 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:04:13.083 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:04:13.084 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:04:13.084 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:04:13.085 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:04:13.085 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:04:13.085 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:04:13.085 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:04:13.799 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:04:13.799 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:04:14.158 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:04:14.512 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:15.222 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:15.223 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:04:15.801 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:04:17.503 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:04:17.872 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:17.979 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:04:18.022 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:04:18.058 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:04:18.090 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:04:18.121 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:04:18.149 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:04:18.175 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:04:18.201 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:04:18.228 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:04:18.256 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:04:18.282 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:04:18.284 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:04:18.284 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:04:18.284 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:04:18.284 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:04:18.285 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:04:18.285 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:04:18.285 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:04:18.285 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:04:18.285 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:04:18.285 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:04:18.285 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:04:18.286 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:04:18.286 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:04:18.725 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:18.725 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:04:18.890 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:04:18.891 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:04:20.435 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:04:20.757 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:20.892 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:04:21.524 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:21.525 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:04:23.749 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:04:23.784 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:04:23.811 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:04:23.836 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:04:23.863 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:04:23.889 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:04:23.915 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:04:23.940 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:04:23.968 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:04:23.983 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:23.993 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:04:24.019 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:04:24.020 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:04:24.021 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:04:24.021 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:04:24.021 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:04:24.021 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:04:24.021 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:04:24.021 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:04:24.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:04:24.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:04:24.022 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:04:24.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:04:24.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:04:24.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:04:24.048 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:04:24.670 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:24.671 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:04:25.133 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:04:25.133 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:04:26.753 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:26.927 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:04:27.134 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:04:27.244 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:27.245 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:04:29.994 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:30.127 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:04:30.150 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:04:30.173 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:04:30.196 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:04:30.222 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:04:30.248 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:04:30.274 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:04:30.301 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:04:30.326 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:04:30.352 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:04:30.378 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:04:30.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:04:30.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:04:30.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:04:30.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:04:30.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:04:30.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:04:30.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:04:30.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:04:30.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:04:30.381 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:04:30.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:04:30.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:04:30.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:04:30.446 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:04:30.499 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:30.500 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:04:31.106 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:04:31.106 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:04:32.744 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:33.107 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:04:33.110 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:04:33.258 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:33.259 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:04:36.075 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:04:36.178 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:36.292 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:04:36.316 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:04:36.340 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:04:36.364 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:04:36.397 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:04:36.442 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:04:36.535 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:04:36.536 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:04:36.559 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:04:36.592 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:04:36.621 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:04:36.623 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:04:36.623 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:04:36.623 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:04:36.624 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:04:36.624 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:04:36.624 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:04:36.624 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:04:36.624 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:04:36.624 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:04:36.624 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:04:36.625 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:04:36.625 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:04:36.625 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:04:36.980 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:36.981 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:04:37.375 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:04:37.375 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:04:38.794 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:04:38.948 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:39.376 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:04:39.458 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:39.459 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:04:42.077 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:04:42.147 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:42.530 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:04:42.575 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:04:42.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:04:42.639 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:04:42.663 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:04:42.690 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:04:42.717 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:04:42.742 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:04:42.769 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:04:42.795 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:04:42.821 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:04:42.823 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:04:42.823 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:04:42.824 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:04:42.824 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:04:42.824 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:04:42.824 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:04:42.824 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:04:42.825 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:04:42.825 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:04:42.825 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:04:42.825 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:04:42.825 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:04:42.825 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:04:42.959 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:42.959 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:04:43.274 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:04:43.274 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:04:44.585 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:04:44.927 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:45.275 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:04:45.570 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:45.571 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:04:48.079 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:04:48.188 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:04:48.211 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:04:48.234 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:04:48.263 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:04:48.289 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:04:48.315 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:04:48.341 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:04:48.368 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:04:48.394 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:04:48.421 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:04:48.448 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:04:48.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:04:48.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:04:48.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:04:48.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:04:48.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:04:48.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:04:48.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:04:48.451 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:04:48.451 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:04:48.451 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:04:48.451 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:04:48.451 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:04:48.451 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:04:48.612 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:49.171 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:04:49.171 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:04:49.342 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:49.342 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:04:50.665 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:04:51.172 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:04:51.240 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:51.922 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:51.923 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:04:54.084 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:04:54.134 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:04:54.159 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:04:54.183 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:04:54.208 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:04:54.234 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:04:54.263 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:04:54.290 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:04:54.340 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:04:54.342 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:04:54.374 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:04:54.403 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:04:54.405 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:04:54.405 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:04:54.405 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:04:54.405 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:04:54.406 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:04:54.406 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:04:54.406 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:04:54.406 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:04:54.406 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:04:54.406 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:04:54.406 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:04:54.407 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:04:54.407 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:04:54.615 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:55.151 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:04:55.151 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:04:55.314 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:55.314 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:04:56.710 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:04:57.152 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:04:57.182 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:04:57.768 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:04:57.768 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:05:00.092 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:05:00.241 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:05:00.265 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:05:00.300 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:05:00.312 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:05:00.340 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:05:00.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:05:00.392 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:05:00.417 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:05:00.445 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:05:00.470 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:05:00.498 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:05:00.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:05:00.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:05:00.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:05:00.500 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:05:00.500 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:05:00.500 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:05:00.500 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:05:00.501 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:05:00.501 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:05:00.501 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:05:00.502 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:05:00.502 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:05:00.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:05:00.666 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:05:00.666 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:05:00.667 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:05:00.667 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:05:01.225 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:05:01.226 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:05:01.391 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:05:02.752 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:05:03.227 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:05:03.233 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:05:03.234 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:05:03.234 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:05:03.234 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:05:03.234 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:05:03.863 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:05:06.336 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:05:06.385 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:05:06.427 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:05:06.450 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:05:06.470 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:05:06.504 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:05:06.537 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:05:06.567 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:05:06.596 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:05:06.623 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:05:06.650 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:05:06.678 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:05:06.678 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:05:06.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:05:06.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:05:06.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:05:06.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:05:06.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:05:06.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:05:06.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:05:06.680 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:05:06.680 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:05:06.680 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:05:06.680 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:05:06.680 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:05:07.126 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:05:07.390 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:05:07.390 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:05:07.755 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:05:09.039 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:05:09.391 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:05:09.710 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:05:10.608 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:05:12.252 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:05:12.272 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:05:12.295 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:05:12.317 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:05:12.343 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:05:12.370 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:05:12.396 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:05:12.422 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:05:12.448 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:05:12.476 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:05:12.503 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:05:12.505 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:05:12.505 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:05:12.505 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:05:12.506 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:05:12.506 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:05:12.506 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:05:12.506 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:05:12.506 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:05:12.506 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:05:12.507 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:05:12.507 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:05:12.507 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:05:12.507 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:05:13.085 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:05:13.091 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:05:13.091 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:05:13.616 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:05:13.616 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:05:13.616 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:05:13.616 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:05:13.616 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:05:14.075 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:05:15.100 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:05:15.666 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:05:16.053 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:05:16.054 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:05:16.054 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:05:16.054 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:05:16.054 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:05:17.214 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:05:17.939 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:05:17.962 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:05:18.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:05:18.021 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:05:18.053 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:05:18.076 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:05:18.105 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:05:18.129 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:05:18.155 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:05:18.181 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:05:18.208 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:05:18.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:05:18.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:05:18.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:05:18.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:05:18.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:05:18.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:05:18.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:05:18.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:05:18.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:05:18.211 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:05:18.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:05:18.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:05:18.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:05:18.904 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:05:18.905 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:05:19.122 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:05:20.035 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:05:20.547 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:05:20.906 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:05:21.683 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:05:23.391 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:05:23.917 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:05:23.917 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:05:23.917 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:05:23.917 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:05:23.918 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:05:23.941 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:05:23.961 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:05:24.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:05:24.032 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:05:24.065 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:05:24.093 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:05:24.095 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:05:24.095 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:05:24.095 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:05:24.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:05:24.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:05:24.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:05:24.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:05:24.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:05:24.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:05:24.096 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:05:24.097 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:05:24.097 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:05:24.097 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:05:24.121 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:05:24.704 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:05:24.705 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:05:25.107 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:05:26.509 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:05:26.509 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:05:26.509 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:05:26.510 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:05:26.510 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:05:26.705 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:05:27.017 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:05:27.849 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:05:29.027 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:05:29.050 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:05:29.076 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:05:29.101 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:05:29.128 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:05:29.155 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:05:29.181 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:05:29.207 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:05:29.233 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:05:29.260 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:05:29.287 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:05:29.289 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:05:29.289 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:05:29.289 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:05:29.289 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:05:29.289 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:05:29.289 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:05:29.290 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:05:29.290 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:05:29.290 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:05:29.290 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:05:29.290 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:05:29.290 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:05:29.290 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:05:29.604 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:05:29.909 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:05:29.909 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:05:30.368 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:05:31.105 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:05:31.910 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:05:32.605 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:05:33.328 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:05:33.808 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:05:34.101 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:05:34.126 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:05:34.150 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:05:34.174 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:05:34.206 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:05:34.236 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:05:34.263 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:05:34.290 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:05:34.317 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:05:34.344 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:05:34.370 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:05:34.372 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:05:34.372 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:05:34.372 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:05:34.373 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:05:34.373 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:05:34.373 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:05:34.373 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:05:34.374 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:05:34.374 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:05:34.374 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:05:34.374 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:05:34.374 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:05:34.375 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:05:35.317 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:05:35.318 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:05:36.200 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:05:36.201 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:05:36.201 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:05:36.201 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:05:36.201 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:05:36.701 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:05:37.159 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:05:37.319 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:05:39.161 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:05:39.737 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:05:39.749 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:05:39.750 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:05:40.122 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:05:40.144 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:05:40.167 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:05:40.189 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:05:40.215 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:05:40.259 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:05:40.300 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:05:40.332 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:05:40.381 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:05:40.396 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:05:40.426 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:05:40.429 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:05:40.429 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:05:40.429 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:05:40.430 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:05:40.430 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:05:40.430 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:05:40.430 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:05:40.430 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:05:40.430 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:05:40.430 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:05:40.430 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:05:40.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:05:40.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:05:41.069 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:05:41.069 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:05:42.271 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:05:42.777 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:05:42.778 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:05:43.070 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:05:43.104 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:05:45.498 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:05:45.835 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:05:45.968 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:05:45.968 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:05:48.263 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:05:48.817 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:05:48.817 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:05:49.116 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:05:51.072 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:05:51.551 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:05:51.551 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:05:51.733 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:05:53.825 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:05:54.334 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:05:54.334 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:05:55.122 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:05:55.620 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:05:55.656 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:05:55.688 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:05:55.716 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:05:55.746 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:05:55.793 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:05:55.829 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:05:55.862 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:05:55.895 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:05:55.924 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:05:55.952 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:05:55.954 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:05:55.954 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:05:55.954 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:05:55.954 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:05:55.954 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:05:55.954 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:05:55.954 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:05:55.955 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:05:55.955 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:05:55.955 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:05:55.955 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:05:55.955 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:05:55.955 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:05:56.530 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:05:56.530 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:05:56.604 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:05:57.104 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:05:57.105 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:05:57.824 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:05:58.531 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:05:59.354 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:06:00.134 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:06:00.134 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:06:01.788 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:06:01.819 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:06:01.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:06:01.872 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:06:01.899 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:06:01.926 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:06:01.969 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:06:02.002 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:06:02.029 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:06:02.061 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:06:02.087 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:06:02.089 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:06:02.089 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:06:02.090 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:06:02.090 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:06:02.090 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:06:02.090 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:06:02.090 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:06:02.090 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:06:02.091 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:06:02.092 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:06:02.092 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:06:02.092 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:06:02.092 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:06:02.101 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:06:02.220 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:06:02.987 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:06:02.988 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:06:03.155 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:06:03.155 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:06:04.906 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:06:04.989 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:06:05.088 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:06:06.008 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:06:06.009 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:06:07.934 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:06:07.958 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:06:07.987 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:06:08.026 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:06:08.062 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:06:08.094 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:06:08.122 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:06:08.123 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:06:08.150 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:06:08.176 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:06:08.204 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:06:08.230 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:06:08.231 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:06:08.231 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:06:08.231 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:06:08.231 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:06:08.231 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:06:08.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:06:08.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:06:08.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:06:08.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:06:08.232 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:06:08.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:06:08.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:06:08.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:06:08.864 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:06:09.109 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:06:09.109 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:06:09.460 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:06:09.460 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:06:10.922 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:06:11.110 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:06:11.447 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:06:12.129 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:06:12.129 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:06:13.899 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:06:13.921 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:06:13.945 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:06:13.969 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:06:14.002 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:06:14.023 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:06:14.049 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:06:14.075 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:06:14.101 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:06:14.128 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:06:14.148 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:06:14.155 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:06:14.155 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:06:14.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:06:14.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:06:14.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:06:14.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:06:14.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:06:14.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:06:14.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:06:14.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:06:14.157 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:06:14.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:06:14.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:06:14.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:06:15.138 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:06:15.138 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:06:15.413 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:06:15.932 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:06:15.932 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:06:16.783 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:06:17.140 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:06:18.499 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:06:19.298 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:06:19.298 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:06:20.109 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:06:20.475 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:06:20.952 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:06:21.002 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:06:21.042 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:06:21.079 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:06:21.112 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:06:21.146 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:06:21.174 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:06:21.200 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:06:21.228 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:06:21.255 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:06:21.257 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:06:21.257 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:06:21.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:06:21.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:06:21.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:06:21.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:06:21.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:06:21.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:06:21.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:06:21.259 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:06:21.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:06:21.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:06:21.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:06:21.792 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:06:21.862 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:06:21.862 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:06:22.265 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:06:22.265 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:06:23.274 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:06:23.863 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:06:24.210 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:06:25.118 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:06:25.118 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:06:26.526 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:06:26.693 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:06:26.709 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:06:26.732 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:06:26.755 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:06:26.786 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:06:26.817 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:06:26.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:06:26.876 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:06:26.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:06:26.931 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:06:26.957 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:06:26.959 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:06:26.959 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:06:26.959 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:06:26.959 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:06:26.960 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:06:26.960 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:06:26.960 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:06:26.960 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:06:26.960 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:06:26.960 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:06:26.960 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:06:26.961 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:06:26.961 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:06:27.783 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:06:27.784 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:06:28.205 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:06:28.711 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:06:28.711 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:06:29.520 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:06:29.785 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:06:31.672 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:06:32.459 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:06:32.460 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:06:32.545 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:06:32.576 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:06:32.604 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:06:32.644 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:06:32.662 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:06:32.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:06:32.716 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:06:32.742 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:06:32.769 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:06:32.797 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:06:32.823 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:06:32.824 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:06:32.824 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:06:32.824 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:06:32.824 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:06:32.825 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:06:32.825 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:06:32.825 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:06:32.825 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:06:32.825 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:06:32.825 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:06:32.825 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:06:32.826 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:06:32.826 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:06:33.131 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:06:33.454 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:06:33.455 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:06:34.645 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:06:35.153 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:06:35.153 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:06:35.456 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:06:36.006 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:06:37.846 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:06:38.446 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:06:38.474 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:06:38.503 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:06:38.526 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:06:38.552 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:06:38.578 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:06:38.585 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:06:38.585 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:06:38.620 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:06:38.653 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:06:38.691 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:06:38.719 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:06:38.746 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:06:38.747 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:06:38.747 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:06:38.747 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:06:38.748 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:06:38.748 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:06:38.748 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:06:38.748 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:06:38.748 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:06:38.748 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:06:38.749 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:06:38.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:06:38.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:06:38.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:06:39.447 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:06:39.447 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:06:40.169 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:06:40.793 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:06:41.223 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:06:41.223 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:06:41.448 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:06:42.851 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:06:43.508 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:06:44.186 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:06:44.211 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:06:44.236 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:06:44.249 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:06:44.261 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:06:44.288 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:06:44.314 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:06:44.341 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:06:44.369 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:06:44.394 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:06:44.421 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:06:44.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:06:44.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:06:44.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:06:44.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:06:44.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:06:44.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:06:44.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:06:44.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:06:44.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:06:44.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:06:44.451 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:06:44.451 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:06:44.451 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:06:44.451 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:06:45.144 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:06:45.144 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:06:46.151 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:06:47.078 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:06:47.078 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:06:47.078 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:06:47.078 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:06:47.078 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:06:47.145 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:06:47.582 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:06:48.752 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:06:49.510 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:06:49.533 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:06:49.560 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:06:49.588 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:06:49.613 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:06:49.644 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:06:49.670 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:06:49.697 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:06:49.724 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:06:49.751 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:06:49.793 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:06:49.795 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:06:49.795 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:06:49.795 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:06:49.796 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:06:49.796 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:06:49.796 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:06:49.796 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:06:49.796 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:06:49.796 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:06:49.796 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:06:49.797 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:06:49.797 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:06:49.797 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:06:50.254 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:06:50.774 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:06:50.775 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:06:51.068 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:06:52.486 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:06:52.775 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:06:53.159 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:06:53.853 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:06:55.126 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:06:55.312 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:06:55.338 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:06:55.372 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:06:55.407 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:06:55.434 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:06:55.461 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:06:55.488 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:06:55.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:06:55.570 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:06:55.601 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:06:55.634 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:06:55.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:06:55.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:06:55.635 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:06:55.635 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:06:55.635 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:06:55.635 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:06:55.635 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:06:55.635 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:06:55.636 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:06:55.636 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:06:55.636 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:06:55.636 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:06:55.636 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:06:56.459 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:06:56.460 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:06:56.533 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:06:56.533 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:06:56.533 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:06:56.533 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:06:56.533 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:06:57.269 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:06:58.460 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:06:58.517 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:06:59.932 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:06:59.933 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:06:59.933 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:06:59.933 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:06:59.933 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:07:00.641 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:07:01.050 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:07:01.073 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:07:01.088 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:07:01.096 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:07:01.119 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:07:01.145 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:07:01.178 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:07:01.204 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:07:01.231 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:07:01.257 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:07:01.289 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:07:01.316 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:07:01.317 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:07:01.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:07:01.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:07:01.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:07:01.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:07:01.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:07:01.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:07:01.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:07:01.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:07:01.319 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:07:01.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:07:01.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:07:01.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:07:02.171 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:07:02.171 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:07:03.356 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:07:03.842 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:07:04.151 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:07:04.172 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:07:06.598 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:07:06.711 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:07:06.903 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:07:07.069 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:07:07.564 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:07:07.610 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:07:07.648 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:07:07.683 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:07:07.716 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:07:07.749 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:07:07.777 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:07:07.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:07:07.838 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:07:07.864 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:07:07.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:07:07.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:07:07.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:07:07.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:07:07.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:07:07.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:07:07.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:07:07.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:07:07.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:07:07.867 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:07:07.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:07:07.868 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:07:07.868 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:07:09.004 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:07:09.005 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:07:09.659 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:07:09.659 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:07:09.659 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:07:09.659 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:07:09.659 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:07:10.157 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:07:10.481 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:07:11.005 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:07:12.820 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:07:12.820 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:07:12.820 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:07:12.820 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:07:13.076 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:07:13.527 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:07:13.559 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:07:13.571 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:07:13.584 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:07:13.608 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:07:13.635 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:07:13.660 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:07:13.707 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:07:13.756 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:07:13.795 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:07:13.824 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:07:13.861 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:07:13.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:07:13.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:07:13.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:07:13.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:07:13.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:07:13.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:07:13.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:07:13.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:07:13.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:07:13.864 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:07:13.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:07:13.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:07:13.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:07:14.833 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:07:14.834 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:07:16.078 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:07:16.530 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:07:16.583 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:07:16.835 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:07:18.872 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:07:18.980 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:07:19.096 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:07:19.118 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:07:19.140 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:07:19.164 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:07:19.191 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:07:19.217 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:07:19.244 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:07:19.269 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:07:19.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:07:19.323 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:07:19.348 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:07:19.350 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:07:19.350 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:07:19.350 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:07:19.351 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:07:19.351 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:07:19.351 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:07:19.351 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:07:19.351 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:07:19.351 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:07:19.351 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:07:19.352 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:07:19.352 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:07:19.352 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:07:19.402 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:07:20.088 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:07:20.089 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:07:21.849 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:07:22.090 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:07:22.176 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:07:22.401 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:07:22.401 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:07:24.841 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:07:24.893 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:07:24.905 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:07:24.920 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:07:24.944 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:07:24.967 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:07:24.994 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:07:25.020 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:07:25.047 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:07:25.074 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:07:25.100 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:07:25.128 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:07:25.154 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:07:25.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:07:25.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:07:25.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:07:25.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:07:25.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:07:25.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:07:25.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:07:25.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:07:25.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:07:25.158 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:07:25.158 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:07:25.158 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:07:25.158 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:07:25.524 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:07:25.525 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:07:25.893 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:07:25.894 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:07:27.835 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:07:27.895 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:07:28.187 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:07:28.396 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:07:28.396 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:07:30.579 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:07:30.603 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:07:30.627 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:07:30.651 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:07:30.677 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:07:30.703 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:07:30.729 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:07:30.755 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:07:30.782 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:07:30.808 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:07:30.835 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:07:30.845 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:07:30.845 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:07:30.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:07:30.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:07:30.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:07:30.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:07:30.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:07:30.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:07:30.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:07:30.848 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:07:30.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:07:30.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:07:30.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:07:30.882 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:07:30.965 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:07:31.494 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:07:31.494 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:07:31.754 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:07:31.771 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:07:33.497 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:07:34.172 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:07:34.175 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:07:34.856 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:07:34.857 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:07:36.167 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:07:36.199 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:07:36.228 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:07:36.254 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:07:36.282 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:07:36.309 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:07:36.347 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:07:36.384 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:07:36.419 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:07:36.459 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:07:36.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:07:36.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:07:36.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:07:36.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:07:36.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:07:36.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:07:36.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:07:36.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:07:36.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:07:36.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:07:36.489 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:07:36.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:07:36.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:07:36.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:07:37.093 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:07:37.093 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:07:37.149 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:07:37.386 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:07:38.099 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:07:38.100 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:07:39.094 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:07:40.483 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:07:40.853 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:07:41.685 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:07:41.686 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:07:43.422 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:07:43.705 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:07:44.360 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:07:44.360 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:07:46.482 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:07:47.045 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:07:47.513 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:07:47.514 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:07:49.012 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:07:49.480 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:07:50.051 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:07:50.051 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:07:52.057 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:07:52.098 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:07:52.135 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:07:52.185 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:07:52.214 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:07:52.242 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:07:52.273 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:07:52.300 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:07:52.327 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:07:52.354 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:07:52.384 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:07:52.411 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:07:52.437 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:07:52.439 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:07:52.439 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:07:52.439 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:07:52.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:07:52.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:07:52.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:07:52.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:07:52.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:07:52.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:07:52.440 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:07:52.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:07:52.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:07:52.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:07:52.850 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:07:52.851 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:07:53.123 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:07:53.123 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:07:54.662 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:07:54.956 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:07:55.124 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:07:55.168 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:07:55.169 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:07:58.189 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:07:58.196 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:07:58.230 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:07:58.256 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:07:58.284 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:07:58.310 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:07:58.337 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:07:58.363 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:07:58.389 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:07:58.417 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:07:58.421 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:07:58.444 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:07:58.470 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:07:58.471 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:07:58.472 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:07:58.472 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:07:58.472 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:07:58.472 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:07:58.472 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:07:58.472 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:07:58.473 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:07:58.473 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:07:58.473 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:07:58.473 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:07:58.473 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:07:58.473 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:07:59.227 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:07:59.227 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:07:59.359 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:07:59.359 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:08:00.790 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:08:01.204 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:08:01.228 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:08:01.770 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:08:01.771 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:08:04.211 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:08:04.234 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:08:04.257 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:08:04.280 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:08:04.308 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:08:04.334 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:08:04.360 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:08:04.386 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:08:04.413 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:08:04.440 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:08:04.466 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:08:04.467 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:08:04.467 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:08:04.468 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:08:04.468 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:08:04.468 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:08:04.468 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:08:04.468 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:08:04.468 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:08:04.468 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:08:04.469 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:08:04.469 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:08:04.469 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:08:04.469 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:08:04.533 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:08:05.095 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:08:05.096 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:08:05.222 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:08:05.265 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:08:05.265 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:08:07.097 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:08:07.269 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:08:07.671 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:08:07.869 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:08:07.869 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:08:10.276 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:08:10.783 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:08:11.136 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:08:11.188 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:08:11.231 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:08:11.267 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:08:11.267 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:08:11.273 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:08:11.304 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:08:11.352 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:08:11.383 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:08:11.415 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:08:11.451 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:08:11.484 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:08:11.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:08:11.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:08:11.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:08:11.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:08:11.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:08:11.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:08:11.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:08:11.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:08:11.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:08:11.490 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:08:11.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:08:11.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:08:11.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:08:11.491 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:08:12.445 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:08:12.446 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:08:13.485 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:08:13.990 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:08:13.991 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:08:14.279 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:08:14.447 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:08:16.812 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:08:17.487 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:08:17.488 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:08:17.668 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:08:17.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:08:17.713 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:08:17.737 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:08:17.762 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:08:17.788 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:08:17.814 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:08:17.866 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:08:17.906 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:08:17.935 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:08:17.963 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:08:17.965 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:08:17.965 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:08:17.965 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:08:17.965 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:08:17.965 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:08:17.965 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:08:17.965 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:08:17.966 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:08:17.966 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:08:17.966 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:08:17.966 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:08:17.966 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:08:17.966 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:08:18.218 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:08:18.572 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:08:18.573 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:08:19.844 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:08:20.349 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:08:20.349 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:08:20.573 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:08:21.024 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:08:22.716 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:08:23.498 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:08:23.525 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:08:23.539 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:08:23.540 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:08:23.564 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:08:23.600 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:08:23.633 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:08:23.663 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:08:23.690 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:08:23.716 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:08:23.742 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:08:23.770 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:08:23.796 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:08:23.797 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:08:23.798 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:08:23.798 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:08:23.798 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:08:23.798 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:08:23.798 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:08:23.798 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:08:23.799 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:08:23.799 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:08:23.799 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:08:23.799 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:08:23.799 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:08:23.799 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:08:24.591 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:08:24.591 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:08:25.208 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:08:25.765 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:08:26.253 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:08:26.593 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:08:27.700 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:08:28.782 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:08:29.364 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:08:29.431 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:08:30.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:08:30.316 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:08:30.342 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:08:30.370 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:08:30.396 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:08:30.423 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:08:30.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:08:30.473 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:08:30.501 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:08:30.528 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:08:30.530 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:08:30.530 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:08:30.530 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:08:30.530 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:08:30.531 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:08:30.531 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:08:30.531 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:08:30.531 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:08:30.532 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:08:30.532 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:08:30.532 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:08:30.532 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:08:30.533 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:08:31.232 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:08:31.232 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:08:31.830 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:08:31.830 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:08:31.830 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:08:31.831 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:08:31.831 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:08:32.351 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:08:33.233 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:08:34.463 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:08:34.463 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:08:34.463 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:08:34.463 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:08:35.193 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:08:35.651 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:08:35.674 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:08:35.697 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:08:35.722 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:08:35.749 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:08:35.775 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:08:35.802 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:08:35.833 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:08:35.859 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:08:35.886 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:08:35.913 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:08:35.919 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:08:35.919 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:08:35.919 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:08:35.932 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:08:35.932 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:08:35.932 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:08:35.933 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:08:35.933 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:08:35.933 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:08:35.933 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:08:35.933 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:08:35.933 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:08:35.934 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:08:36.701 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:08:36.701 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:08:37.284 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:08:37.778 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:08:38.702 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:08:40.093 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:08:40.695 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:08:40.797 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:08:40.931 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:08:40.960 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:08:40.980 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:08:41.015 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:08:41.050 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:08:41.104 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:08:41.145 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:08:41.182 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:08:41.217 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:08:41.250 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:08:41.281 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:08:41.283 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:08:41.283 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:08:41.283 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:08:41.284 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:08:41.284 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:08:41.284 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:08:41.284 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:08:41.284 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:08:41.284 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:08:41.285 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:08:41.285 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:08:41.285 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:08:41.285 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:08:41.827 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:08:41.827 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:08:43.151 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:08:43.151 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:08:43.151 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:08:43.152 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:08:43.396 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:08:43.621 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:08:43.828 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:08:46.432 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:08:46.433 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:08:46.433 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:08:46.433 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:08:46.433 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:08:46.490 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:08:46.540 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:08:46.563 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:08:46.586 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:08:46.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:08:46.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:08:46.666 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:08:46.692 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:08:46.719 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:08:46.750 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:08:46.794 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:08:46.834 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:08:46.836 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:08:46.836 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:08:46.836 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:08:46.836 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:08:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:08:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:08:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:08:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:08:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:08:46.837 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:08:46.838 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:08:46.838 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:08:46.838 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:08:47.103 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:08:47.554 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:08:47.554 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:08:49.160 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:08:49.392 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:08:49.555 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:08:49.644 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:08:52.521 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:08:52.677 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:08:52.700 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:08:52.722 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:08:52.745 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:08:52.771 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:08:52.798 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:08:52.824 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:08:52.851 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:08:52.877 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:08:52.916 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:08:52.970 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:08:52.973 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:08:52.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:08:52.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:08:52.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:08:52.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:08:52.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:08:52.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:08:52.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:08:52.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:08:52.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:08:52.975 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:08:52.976 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:08:52.976 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:08:52.976 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:08:53.209 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:08:53.572 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:08:53.572 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:08:55.300 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:08:55.300 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:08:55.300 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:08:55.301 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:08:55.301 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:08:55.573 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:08:55.818 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:08:55.980 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:08:58.248 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:08:58.270 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:08:58.294 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:08:58.318 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:08:58.344 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:08:58.389 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:08:58.420 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:08:58.450 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:08:58.479 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:08:58.509 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:08:58.537 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:08:58.539 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:08:58.539 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:08:58.539 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:08:58.539 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:08:58.539 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:08:58.539 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:08:58.540 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:08:58.540 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:08:58.540 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:08:58.540 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:08:58.540 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:08:58.540 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:08:58.541 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:08:58.812 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:08:59.438 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:08:59.438 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:08:59.450 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:08:59.614 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:09:01.439 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:09:01.582 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:09:01.960 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:09:02.087 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:09:02.088 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:09:04.848 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:09:04.869 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:09:04.893 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:09:04.916 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:09:04.943 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:09:04.971 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:09:04.997 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:09:05.024 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:09:05.051 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:09:05.078 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:09:05.105 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:09:05.106 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:09:05.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:09:05.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:09:05.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:09:05.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:09:05.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:09:05.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:09:05.108 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:09:05.108 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:09:05.108 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:09:05.108 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:09:05.108 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:09:05.108 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:09:05.227 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:09:05.298 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:09:05.840 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:09:05.841 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:09:06.221 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:09:06.222 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:09:07.841 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:09:08.219 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:09:08.222 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:09:09.058 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:09:09.059 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:09:10.922 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:09:10.958 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:09:10.990 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:09:11.019 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:09:11.042 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:09:11.072 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:09:11.098 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:09:11.125 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:09:11.151 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:09:11.180 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:09:11.206 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:09:11.207 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:09:11.207 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:09:11.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:09:11.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:09:11.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:09:11.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:09:11.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:09:11.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:09:11.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:09:11.209 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:09:11.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:09:11.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:09:11.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:09:11.813 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:09:11.935 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:09:11.954 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:09:11.954 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:09:12.422 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:09:12.423 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:09:13.955 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:09:14.324 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:09:14.566 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:09:15.061 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:09:15.061 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:09:16.677 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:09:16.708 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:09:16.737 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:09:16.764 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:09:16.789 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:09:16.817 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:09:16.843 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:09:16.874 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:09:16.896 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:09:16.940 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:09:16.977 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:09:16.980 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:09:16.980 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:09:16.980 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:09:16.980 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:09:16.980 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:09:16.981 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:09:16.981 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:09:16.981 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:09:16.981 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:09:16.981 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:09:16.981 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:09:16.981 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:09:16.982 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:09:17.577 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:09:17.577 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:09:17.995 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:09:18.297 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:09:18.502 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:09:18.502 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:09:19.579 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:09:20.381 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:09:20.853 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:09:20.917 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:09:20.917 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:09:22.592 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:09:22.624 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:09:22.656 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:09:22.682 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:09:22.710 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:09:22.736 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:09:22.762 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:09:22.789 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:09:22.816 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:09:22.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:09:22.876 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:09:22.877 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:09:22.878 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:09:22.878 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:09:22.878 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:09:22.878 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:09:22.878 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:09:22.878 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:09:22.879 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:09:22.879 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:09:22.879 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:09:22.879 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:09:22.879 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:09:22.879 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:09:23.603 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:09:23.603 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:09:23.832 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:09:24.303 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:09:24.345 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:09:24.346 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:09:25.605 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:09:26.314 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:09:26.763 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:09:26.879 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:09:26.880 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:09:28.549 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:09:28.573 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:09:28.597 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:09:28.622 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:09:28.650 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:09:28.676 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:09:28.702 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:09:28.728 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:09:28.754 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:09:28.781 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:09:28.810 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:09:28.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:09:28.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:09:28.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:09:28.812 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:09:28.812 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:09:28.812 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:09:28.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:09:28.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:09:28.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:09:28.813 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:09:28.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:09:28.814 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:09:28.814 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:09:29.534 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:09:29.534 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:09:29.679 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:09:30.251 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:09:30.289 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:09:30.290 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:09:31.535 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:09:32.391 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:09:32.829 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:09:33.051 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:09:33.051 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:09:34.370 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:09:34.399 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:09:34.444 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:09:34.478 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:09:34.508 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:09:34.536 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:09:34.562 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:09:34.589 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:09:34.615 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:09:34.642 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:09:34.669 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:09:34.671 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:09:34.671 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:09:34.671 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:09:34.671 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:09:34.671 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:09:34.671 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:09:34.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:09:34.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:09:34.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:09:34.672 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:09:34.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:09:34.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:09:34.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:09:35.506 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:09:35.507 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:09:35.911 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:09:36.245 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:09:36.434 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:09:36.435 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:09:37.508 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:09:38.673 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:09:38.791 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:09:39.325 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:09:39.325 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:09:40.213 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:09:40.213 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:09:40.213 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:09:40.236 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:09:40.269 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:09:40.299 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:09:40.328 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:09:40.354 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:09:40.381 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:09:40.406 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:09:40.432 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:09:40.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:09:40.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:09:40.435 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:09:40.435 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:09:40.435 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:09:40.435 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:09:40.435 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:09:40.435 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:09:40.436 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:09:40.436 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:09:40.436 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:09:40.436 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:09:40.436 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:09:41.444 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:09:41.445 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:09:42.027 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:09:42.503 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:09:42.503 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:09:42.515 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:09:43.445 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:09:44.892 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:09:45.026 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:09:45.506 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:09:45.507 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:09:46.201 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:09:46.223 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:09:46.247 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:09:46.271 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:09:46.297 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:09:46.324 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:09:46.350 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:09:46.377 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:09:46.403 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:09:46.429 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:09:46.456 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:09:46.457 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:09:46.457 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:09:46.457 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:09:46.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:09:46.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:09:46.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:09:46.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:09:46.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:09:46.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:09:46.458 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:09:46.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:09:46.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:09:46.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:09:47.277 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:09:47.278 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:09:48.250 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:09:48.278 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:09:49.008 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:09:49.009 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:09:49.279 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:09:50.866 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:09:51.232 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:09:51.784 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:09:51.785 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:09:51.785 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:09:51.785 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:09:51.795 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:09:51.824 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:09:51.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:09:51.880 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:09:51.919 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:09:51.919 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:09:51.920 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:09:51.957 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:09:51.989 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:09:51.992 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:09:51.992 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:09:51.993 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:09:51.993 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:09:51.993 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:09:51.993 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:09:51.993 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:09:51.993 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:09:51.994 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:09:51.994 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:09:51.994 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:09:51.994 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:09:51.994 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:09:53.040 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:09:53.040 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:09:54.140 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:09:54.260 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:09:54.783 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:09:54.784 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:09:55.042 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:09:56.946 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:09:56.983 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:09:57.263 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:09:57.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:09:57.313 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:09:57.337 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:09:57.363 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:09:57.393 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:09:57.417 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:09:57.465 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:09:57.508 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:09:57.543 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:09:57.575 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:09:57.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:09:57.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:09:57.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:09:57.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:09:57.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:09:57.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:09:57.578 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:09:57.578 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:09:57.578 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:09:57.578 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:09:57.578 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:09:57.578 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:09:57.579 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:09:57.752 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:09:57.753 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:09:58.269 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:09:58.270 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:10:00.146 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:10:00.271 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:10:00.316 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:10:00.839 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:10:00.840 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:10:03.071 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:10:03.095 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:10:03.102 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:10:03.123 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:10:03.154 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:10:03.177 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:10:03.205 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:10:03.231 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:10:03.241 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:10:03.258 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:10:03.285 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:10:03.331 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:10:03.357 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:10:03.358 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:10:03.359 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:10:03.359 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:10:03.359 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:10:03.359 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:10:03.360 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:10:03.360 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:10:03.360 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:10:03.360 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:10:03.361 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:10:03.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:10:03.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:10:03.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:10:04.089 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:10:04.089 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:10:04.257 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:10:04.257 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:10:06.258 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:10:06.510 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:10:06.517 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:10:06.518 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:10:06.518 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:10:06.518 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:10:07.518 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:10:08.725 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:10:08.748 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:10:08.773 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:10:08.796 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:10:08.825 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:10:08.851 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:10:08.877 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:10:08.903 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:10:08.930 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:10:08.957 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:10:08.983 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:10:08.985 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:10:08.985 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:10:08.985 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:10:08.985 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:10:08.986 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:10:08.986 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:10:08.986 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:10:08.986 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:10:08.986 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:10:08.986 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:10:08.986 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:10:08.987 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:10:08.987 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:10:09.290 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:10:09.630 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:10:09.630 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:10:09.920 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:10:10.439 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:10:11.631 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:10:12.272 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:10:12.411 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:10:13.197 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:10:14.366 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:10:14.389 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:10:14.413 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:10:14.436 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:10:14.464 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:10:14.489 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:10:14.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:10:14.541 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:10:14.590 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:10:14.632 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:10:14.681 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:10:14.682 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:10:14.682 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:10:14.682 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:10:14.682 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:10:14.682 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:10:14.682 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:10:14.683 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:10:14.683 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:10:14.683 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:10:14.683 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:10:14.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:10:14.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:10:14.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:10:15.028 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:10:15.288 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:10:15.289 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:10:15.629 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:10:15.629 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:10:15.629 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:10:15.629 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:10:16.239 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:10:17.290 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:10:18.331 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:10:18.451 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:10:18.451 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:10:18.451 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:10:18.451 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:10:18.451 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:10:19.528 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:10:19.916 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:10:19.938 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:10:19.961 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:10:19.982 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:10:20.008 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:10:20.034 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:10:20.060 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:10:20.089 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:10:20.116 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:10:20.166 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:10:20.208 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:10:20.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:10:20.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:10:20.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:10:20.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:10:20.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:10:20.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:10:20.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:10:20.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:10:20.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:10:20.211 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:10:20.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:10:20.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:10:20.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:10:20.920 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:10:20.920 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:10:21.259 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:10:21.647 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:10:22.213 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:10:22.921 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:10:24.273 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:10:24.798 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:10:25.648 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:10:26.022 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:10:26.047 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:10:26.071 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:10:26.094 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:10:26.121 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:10:26.147 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:10:26.174 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:10:26.201 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:10:26.228 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:10:26.255 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:10:26.283 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:10:26.286 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:10:26.286 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:10:26.286 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:10:26.286 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:10:26.286 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:10:26.287 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:10:26.287 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:10:26.287 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:10:26.287 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:10:26.287 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:10:26.287 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:10:26.287 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:10:26.288 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:10:26.993 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:10:26.993 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:10:27.143 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:10:27.784 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:10:27.784 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:10:27.785 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:10:27.785 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:10:27.785 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:10:28.318 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:10:28.994 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:10:30.517 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:10:30.982 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:10:30.982 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:10:30.982 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:10:30.983 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:10:30.983 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:10:31.714 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:10:32.069 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:10:32.093 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:10:32.118 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:10:32.144 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:10:32.170 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:10:32.213 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:10:32.241 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:10:32.271 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:10:32.301 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:10:32.340 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:10:32.373 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:10:32.374 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:10:32.375 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:10:32.375 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:10:32.375 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:10:32.376 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:10:32.376 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:10:32.376 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:10:32.376 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:10:32.377 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:10:32.378 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:10:32.378 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:10:32.378 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:10:32.378 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:10:33.249 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:10:33.250 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:10:33.250 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:10:33.889 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:10:34.464 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:10:35.252 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:10:36.518 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:10:37.001 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:10:37.757 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:10:38.146 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:10:38.170 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:10:38.194 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:10:38.218 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:10:38.243 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:10:38.269 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:10:38.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:10:38.337 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:10:38.369 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:10:38.401 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:10:38.434 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:10:38.436 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:10:38.437 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:10:38.437 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:10:38.437 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:10:38.437 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:10:38.437 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:10:38.437 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:10:38.438 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:10:38.438 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:10:38.438 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:10:38.438 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:10:38.438 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:10:38.438 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:10:39.225 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:10:39.226 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:10:39.307 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:10:39.945 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:10:39.945 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:10:39.946 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:10:39.946 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:10:39.946 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:10:40.518 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:10:41.227 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:10:42.088 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:10:43.152 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:10:43.755 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:10:43.755 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:10:44.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:10:44.799 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:10:44.854 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:10:44.899 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:10:44.931 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:10:44.962 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:10:44.997 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:10:45.025 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:10:45.051 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:10:45.079 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:10:45.104 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:10:45.130 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:10:45.132 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:10:45.132 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:10:45.132 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:10:45.132 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:10:45.133 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:10:45.133 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:10:45.133 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:10:45.133 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:10:45.133 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:10:45.134 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:10:45.134 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:10:45.134 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:10:45.134 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:10:45.799 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:10:45.943 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:10:45.944 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:10:46.509 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:10:46.510 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:10:47.944 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:10:48.280 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:10:48.526 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:10:49.296 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:10:49.296 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:10:50.652 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:10:50.695 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:10:50.725 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:10:50.751 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:10:50.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:10:50.804 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:10:50.831 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:10:50.856 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:10:50.883 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:10:50.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:10:50.936 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:10:50.938 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:10:50.938 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:10:50.938 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:10:50.938 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:10:50.938 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:10:50.938 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:10:50.938 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:10:50.939 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:10:50.939 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:10:50.939 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:10:50.939 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:10:50.939 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:10:50.939 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:10:51.119 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:10:51.338 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:10:51.480 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:10:51.481 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:10:51.968 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:10:51.969 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:10:53.482 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:10:54.284 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:10:54.912 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:10:55.626 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:10:55.627 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:10:56.034 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:10:56.057 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:10:56.081 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:10:56.105 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:10:56.134 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:10:56.158 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:10:56.187 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:10:56.218 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:10:56.251 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:10:56.283 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:10:56.321 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:10:56.323 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:10:56.323 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:10:56.323 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:10:56.324 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:10:56.324 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:10:56.324 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:10:56.324 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:10:56.324 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:10:56.324 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:10:56.325 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:10:56.325 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:10:56.325 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:10:56.325 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:10:57.056 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:10:57.056 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:10:57.134 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:10:57.860 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:10:58.617 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:10:58.617 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:10:59.057 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:11:00.343 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:11:01.692 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:11:01.972 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:11:02.012 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:11:02.049 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:11:02.081 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:11:02.112 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:11:02.143 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:11:02.170 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:11:02.196 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:11:02.223 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:11:02.249 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:11:02.275 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:11:02.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:11:02.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:11:02.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:11:02.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:11:02.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:11:02.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:11:02.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:11:02.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:11:02.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:11:02.278 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:11:02.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:11:02.279 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:11:02.279 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:11:02.301 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:11:02.302 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:11:02.759 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:11:02.760 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:11:03.262 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:11:04.761 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:11:04.822 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:11:05.457 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:11:05.458 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:11:06.535 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:11:07.760 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:11:07.791 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:11:07.818 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:11:07.841 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:11:07.866 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:11:07.892 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:11:07.916 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:11:07.941 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:11:07.967 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:11:07.986 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:11:08.000 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:11:08.026 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:11:08.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:11:08.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:11:08.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:11:08.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:11:08.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:11:08.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:11:08.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:11:08.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:11:08.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:11:08.029 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:11:08.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:11:08.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:11:08.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:11:08.697 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:11:08.697 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:11:08.839 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:11:08.840 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:11:09.272 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:11:10.841 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:11:11.032 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:11:11.903 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:11:11.904 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:11:13.085 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:11:13.882 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:11:13.913 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:11:13.942 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:11:13.969 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:11:13.994 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:11:14.020 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:11:14.048 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:11:14.074 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:11:14.100 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:11:14.137 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:11:14.171 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:11:14.173 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:11:14.173 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:11:14.174 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:11:14.174 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:11:14.174 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:11:14.174 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:11:14.174 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:11:14.174 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:11:14.174 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:11:14.175 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:11:14.175 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:11:14.175 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:11:14.175 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:11:14.802 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:11:14.849 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:11:14.849 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:11:15.227 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:11:15.228 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:11:16.057 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:11:16.850 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:11:17.319 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:11:18.183 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:11:18.184 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:11:19.399 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:11:20.000 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:11:20.851 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:11:20.863 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:11:20.902 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:11:20.943 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:11:20.980 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:11:21.007 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:11:21.034 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:11:21.059 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:11:21.086 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:11:21.113 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:11:21.149 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:11:21.151 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:11:21.151 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:11:21.151 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:11:21.152 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:11:21.152 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:11:21.152 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:11:21.152 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:11:21.152 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:11:21.152 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:11:21.153 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:11:21.153 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:11:21.153 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:11:21.153 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:11:21.388 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:11:21.388 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:11:21.802 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:11:21.802 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:11:22.296 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:11:23.622 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:11:23.804 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:11:24.388 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:11:24.388 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:11:25.416 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:11:27.057 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:11:27.102 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:11:27.642 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:11:27.643 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:11:27.911 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:11:27.961 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:11:28.003 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:11:28.041 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:11:28.069 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:11:28.099 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:11:28.127 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:11:28.155 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:11:28.183 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:11:28.186 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:11:28.223 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:11:28.225 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:11:28.225 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:11:28.225 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:11:28.225 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:11:28.225 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:11:28.225 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:11:28.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:11:28.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:11:28.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:11:28.226 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:11:28.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:11:28.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:11:28.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:11:28.756 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:11:28.756 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:11:30.101 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:11:30.689 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:11:30.690 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:11:30.757 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:11:32.086 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:11:33.328 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:11:33.537 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:11:34.087 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:11:34.088 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:11:34.189 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:11:34.222 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:11:34.254 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:11:34.282 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:11:34.308 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:11:34.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:11:34.357 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:11:34.384 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:11:34.411 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:11:34.438 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:11:34.439 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:11:34.439 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:11:34.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:11:34.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:11:34.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:11:34.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:11:34.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:11:34.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:11:34.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:11:34.441 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:11:34.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:11:34.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:11:34.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:11:34.869 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:11:35.091 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:11:35.092 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:11:36.408 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:11:37.004 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:11:37.004 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:11:37.095 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:11:38.317 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:11:39.675 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:11:40.195 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:11:40.199 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:11:40.199 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:11:40.223 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:11:40.244 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:11:40.267 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:11:40.292 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:11:40.319 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:11:40.343 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:11:40.367 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:11:40.393 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:11:40.422 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:11:40.448 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:11:40.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:11:40.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:11:40.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:11:40.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:11:40.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:11:40.451 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:11:40.451 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:11:40.451 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:11:40.451 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:11:40.451 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:11:40.451 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:11:40.451 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:11:40.452 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:11:41.035 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:11:41.036 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:11:41.219 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:11:42.437 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:11:43.037 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:11:43.052 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:11:43.052 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:11:44.361 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:11:45.761 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:11:46.247 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:11:46.279 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:11:46.280 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:11:46.280 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:11:46.304 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:11:46.339 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:11:46.373 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:11:46.404 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:11:46.434 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:11:46.462 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:11:46.489 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:11:46.516 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:11:46.542 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:11:46.543 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:11:46.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:11:46.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:11:46.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:11:46.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:11:46.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:11:46.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:11:46.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:11:46.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:11:46.545 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:11:46.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:11:46.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:11:46.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:11:47.116 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:11:47.116 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:11:47.337 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:11:48.820 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:11:49.117 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:11:49.397 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:11:50.533 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:11:51.821 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:11:52.107 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:11:52.108 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:11:52.108 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:11:52.108 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:11:52.108 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:11:52.645 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:11:52.674 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:11:52.745 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:11:52.787 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:11:52.823 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:11:52.857 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:11:52.888 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:11:52.917 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:11:52.969 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:11:53.011 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:11:53.046 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:11:53.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:11:53.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:11:53.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:11:53.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:11:53.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:11:53.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:11:53.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:11:53.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:11:53.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:11:53.050 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:11:53.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:11:53.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:11:53.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:11:53.346 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:11:53.721 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:11:53.722 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:11:55.133 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:11:55.133 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:11:55.133 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:11:55.133 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:11:55.133 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:11:55.722 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:11:55.793 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:11:56.531 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:11:58.306 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:11:58.331 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:11:58.355 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:11:58.380 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:11:58.406 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:11:58.433 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:11:58.460 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:11:58.498 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:11:58.505 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:11:58.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:11:58.566 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:11:58.593 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:11:58.594 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:11:58.595 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:11:58.595 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:11:58.595 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:11:58.595 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:11:58.595 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:11:58.595 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:11:58.596 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:11:58.596 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:11:58.596 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:11:58.596 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:11:58.596 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:11:59.119 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:11:59.133 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:11:59.298 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:11:59.298 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:12:01.300 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:12:01.647 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:12:02.088 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:12:02.247 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:12:04.094 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:12:04.123 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:12:04.150 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:12:04.175 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:12:04.202 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:12:04.228 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:12:04.254 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:12:04.280 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:12:04.306 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:12:04.354 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:12:04.395 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:12:04.397 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:12:04.397 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:12:04.397 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:12:04.398 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:12:04.398 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:12:04.398 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:12:04.398 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:12:04.398 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:12:04.398 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:12:04.398 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:12:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:12:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:12:04.399 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:12:04.712 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:12:04.893 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:12:04.893 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:12:04.894 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:12:04.894 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:12:04.894 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:12:04.931 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:12:04.931 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:12:05.330 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:12:06.932 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:12:07.570 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:12:07.570 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:12:07.570 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:12:07.571 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:12:08.147 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:12:08.330 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:12:09.724 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:12:09.758 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:12:09.790 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:12:09.821 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:12:09.851 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:12:09.879 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:12:09.907 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:12:09.935 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:12:09.962 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:12:09.998 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:12:10.042 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:12:10.044 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:12:10.045 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:12:10.045 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:12:10.045 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:12:10.045 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:12:10.045 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:12:10.045 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:12:10.046 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:12:10.046 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:12:10.046 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:12:10.046 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:12:10.046 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:12:10.046 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:12:10.634 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:12:10.634 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:12:10.905 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:12:11.090 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:12:11.437 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:12:12.636 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:12:14.182 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:12:14.350 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:12:14.826 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:12:15.202 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:12:15.234 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:12:15.254 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:12:15.282 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:12:15.309 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:12:15.335 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:12:15.368 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:12:15.401 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:12:15.426 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:12:15.453 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:12:15.480 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:12:15.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:12:15.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:12:15.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:12:15.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:12:15.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:12:15.483 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:12:15.483 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:12:15.483 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:12:15.484 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:12:15.484 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:12:15.484 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:12:15.484 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:12:15.484 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:12:16.021 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:12:16.021 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:12:17.170 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:12:17.171 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:12:17.171 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:12:17.171 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:12:17.384 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:12:17.700 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:12:18.023 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:12:20.471 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:12:20.471 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:12:20.471 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:12:20.471 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:12:20.471 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:12:21.027 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:12:21.068 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:12:21.091 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:12:21.127 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:12:21.152 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:12:21.178 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:12:21.214 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:12:21.240 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:12:21.267 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:12:21.294 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:12:21.321 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:12:21.348 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:12:21.351 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:12:21.351 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:12:21.351 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:12:21.351 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:12:21.351 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:12:21.352 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:12:21.352 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:12:21.352 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:12:21.352 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:12:21.352 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:12:21.353 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:12:21.353 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:12:21.353 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:12:21.354 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:12:22.178 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:12:22.178 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:12:23.297 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:12:23.791 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:12:24.180 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:12:24.337 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:12:26.794 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:12:26.938 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:12:26.963 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:12:26.993 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:12:27.009 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:12:27.045 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:12:27.065 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:12:27.091 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:12:27.116 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:12:27.143 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:12:27.170 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:12:27.209 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:12:27.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:12:27.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:12:27.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:12:27.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:12:27.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:12:27.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:12:27.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:12:27.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:12:27.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:12:27.210 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:12:27.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:12:27.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:12:27.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:12:27.415 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:12:27.416 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:12:27.994 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:12:27.994 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:12:28.337 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:12:29.819 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:12:29.995 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:12:30.339 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:12:30.339 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:12:31.158 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:12:32.601 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:12:33.091 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:12:33.454 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:12:33.505 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:12:33.541 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:12:33.576 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:12:33.607 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:12:33.636 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:12:33.671 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:12:33.671 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:12:33.694 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:12:33.736 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:12:33.773 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:12:33.809 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:12:33.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:12:33.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:12:33.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:12:33.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:12:33.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:12:33.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:12:33.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:12:33.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:12:33.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:12:33.811 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:12:33.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:12:33.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:12:33.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:12:34.354 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:12:34.541 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:12:34.541 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:12:36.090 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:12:36.542 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:12:36.597 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:12:36.597 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:12:37.072 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:12:39.524 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:12:40.107 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:12:40.107 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:12:40.128 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:12:40.153 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:12:40.178 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:12:40.200 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:12:40.227 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:12:40.254 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:12:40.280 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:12:40.332 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:12:40.359 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:12:40.378 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:12:40.413 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:12:40.446 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:12:40.448 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:12:40.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:12:40.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:12:40.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:12:40.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:12:40.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:12:40.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:12:40.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:12:40.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:12:40.450 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:12:40.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:12:40.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:12:40.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:12:41.460 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:12:41.460 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:12:42.481 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:12:43.006 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:12:43.007 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:12:43.020 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:12:43.461 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:12:45.973 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:12:46.170 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:12:46.486 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:12:46.487 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:12:47.027 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:12:47.075 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:12:47.118 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:12:47.156 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:12:47.197 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:12:47.230 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:12:47.262 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:12:47.290 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:12:47.320 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:12:47.364 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:12:47.366 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:12:47.366 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:12:47.366 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:12:47.366 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:12:47.367 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:12:47.367 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:12:47.367 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:12:47.367 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:12:47.367 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:12:47.367 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:12:47.367 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:12:47.367 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:12:47.368 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:12:48.124 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:12:48.125 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:12:48.700 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:12:49.235 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:12:49.235 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:12:50.126 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:12:51.671 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:12:52.301 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:12:52.301 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:12:52.563 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:12:52.585 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:12:52.609 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:12:52.632 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:12:52.658 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:12:52.684 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:12:52.710 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:12:52.736 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:12:52.763 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:12:52.790 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:12:52.816 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:12:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:12:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:12:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:12:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:12:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:12:52.819 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:12:52.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:12:52.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:12:52.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:12:52.820 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:12:52.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:12:52.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:12:52.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:12:53.371 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:12:53.371 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:12:54.396 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:12:54.883 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:12:54.884 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:12:55.372 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:12:56.539 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:12:57.735 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:12:58.260 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:12:58.260 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:12:58.310 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:12:59.168 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:12:59.210 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:12:59.237 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:12:59.268 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:12:59.297 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:12:59.313 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:12:59.325 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:12:59.353 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:12:59.379 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:12:59.420 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:12:59.457 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:12:59.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:12:59.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:12:59.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:12:59.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:12:59.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:12:59.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:12:59.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:12:59.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:12:59.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:12:59.459 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:12:59.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:12:59.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:12:59.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:13:00.053 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:13:00.053 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:13:00.635 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:13:01.401 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:13:01.402 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:13:02.054 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:13:02.575 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:13:04.149 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:13:04.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:13:04.665 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:13:04.666 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:13:05.432 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:13:05.499 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:13:05.529 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:13:05.559 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:13:05.588 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:13:05.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:13:05.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:13:05.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:13:05.693 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:13:05.719 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:13:05.748 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:13:05.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:13:05.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:13:05.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:13:05.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:13:05.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:13:05.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:13:05.751 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:13:05.751 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:13:05.751 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:13:05.751 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:13:05.751 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:13:05.751 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:13:05.752 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:13:06.450 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:13:06.450 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:13:07.184 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:13:07.745 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:13:07.746 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:13:08.368 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:13:08.452 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:13:10.229 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:13:10.855 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:13:10.856 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:13:10.956 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:13:11.246 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:13:12.106 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:13:12.153 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:13:12.204 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:13:12.258 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:13:12.287 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:13:12.317 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:13:12.346 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:13:12.383 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:13:12.435 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:13:12.461 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:13:12.463 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:13:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:13:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:13:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:13:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:13:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:13:12.464 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:13:12.465 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:13:12.465 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:13:12.465 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:13:12.465 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:13:12.465 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:13:12.465 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:13:13.088 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:13:13.088 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:13:13.359 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:13:13.868 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:13:13.868 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:13:14.366 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:13:15.089 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:13:15.870 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:13:16.380 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:13:16.380 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:13:16.947 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:13:17.490 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:13:17.525 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:13:17.547 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:13:17.572 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:13:17.598 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:13:17.625 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:13:17.652 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:13:17.682 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:13:17.709 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:13:17.740 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:13:17.762 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:13:17.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:13:17.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:13:17.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:13:17.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:13:17.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:13:17.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:13:17.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:13:17.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:13:17.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:13:17.766 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:13:17.767 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:13:17.767 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:13:17.767 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:13:18.472 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:13:18.472 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:13:19.110 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:13:19.615 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:13:19.615 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:13:20.383 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:13:20.473 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:13:22.350 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:13:23.000 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:13:23.001 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:13:23.080 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:13:23.319 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:13:23.342 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:13:23.366 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:13:23.390 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:13:23.416 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:13:23.442 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:13:23.468 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:13:23.495 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:13:23.528 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:13:23.548 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:13:23.575 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:13:23.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:13:23.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:13:23.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:13:23.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:13:23.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:13:23.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:13:23.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:13:23.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:13:23.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:13:23.577 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:13:23.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:13:23.578 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:13:23.578 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:13:24.430 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:13:24.430 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:13:25.474 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:13:26.060 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:13:26.060 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:13:26.431 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:13:26.546 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:13:28.460 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:13:28.889 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:13:28.889 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:13:29.120 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:13:29.431 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:13:29.454 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:13:29.478 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:13:29.503 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:13:29.530 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:13:29.559 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:13:29.587 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:13:29.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:13:29.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:13:29.679 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:13:29.693 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:13:29.696 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:13:29.696 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:13:29.697 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:13:29.697 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:13:29.697 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:13:29.697 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:13:29.697 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:13:29.697 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:13:29.698 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:13:29.698 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:13:29.698 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:13:29.698 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:13:29.698 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:13:30.685 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:13:30.686 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:13:31.532 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:13:31.533 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:13:31.533 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:13:31.533 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:13:32.013 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:13:32.486 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:13:32.686 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:13:34.653 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:13:34.654 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:13:34.654 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:13:34.654 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:13:34.654 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:13:35.161 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:13:35.315 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:13:35.379 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:13:35.418 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:13:35.463 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:13:35.508 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:13:35.542 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:13:35.567 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:13:35.593 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:13:35.620 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:13:35.647 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:13:35.674 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:13:35.703 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:13:35.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:13:35.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:13:35.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:13:35.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:13:35.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:13:35.706 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:13:35.706 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:13:35.706 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:13:35.706 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:13:35.706 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:13:35.706 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:13:35.706 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:13:35.707 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:13:36.376 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:13:36.376 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:13:38.028 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:13:38.378 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:13:38.393 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:13:38.529 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:13:40.950 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:13:40.960 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:13:41.281 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:13:41.282 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:13:41.282 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:13:41.282 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:13:41.823 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:13:41.873 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:13:41.911 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:13:41.944 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:13:41.997 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:13:42.032 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:13:42.037 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:13:42.073 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:13:42.104 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:13:42.132 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:13:42.158 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:13:42.159 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:13:42.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:13:42.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:13:42.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:13:42.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:13:42.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:13:42.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:13:42.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:13:42.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:13:42.161 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:13:42.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:13:42.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:13:42.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:13:42.968 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:13:42.969 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:13:44.418 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:13:44.421 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:13:44.421 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:13:44.422 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:13:44.422 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:13:44.422 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:13:44.970 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:13:45.183 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:13:47.054 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:13:47.220 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:13:47.242 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:13:47.271 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:13:47.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:13:47.324 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:13:47.350 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:13:47.392 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:13:47.410 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:13:47.436 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:13:47.462 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:13:47.489 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:13:47.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:13:47.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:13:47.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:13:47.491 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:13:47.491 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:13:47.491 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:13:47.491 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:13:47.491 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:13:47.491 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:13:47.491 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:13:47.492 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:13:47.492 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:13:47.492 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:13:47.515 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:13:48.226 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:13:48.509 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:13:48.510 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:13:50.488 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:13:50.511 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:13:50.550 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:13:51.101 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:13:53.223 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:13:53.253 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:13:53.280 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:13:53.304 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:13:53.331 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:13:53.358 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:13:53.384 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:13:53.398 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:13:53.410 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:13:53.438 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:13:53.466 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:13:53.473 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:13:53.473 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:13:53.474 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:13:53.474 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:13:53.474 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:13:53.493 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:13:53.494 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:13:53.494 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:13:53.494 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:13:53.494 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:13:53.495 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:13:53.495 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:13:53.497 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:13:53.497 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:13:53.497 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:13:53.498 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:13:53.498 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:13:53.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:13:53.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:13:54.093 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:13:54.350 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:13:54.351 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:13:56.326 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:13:56.326 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:13:56.326 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:13:56.326 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:13:56.326 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:13:56.351 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:13:56.551 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:13:56.817 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:13:59.211 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:13:59.357 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:13:59.408 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:13:59.408 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:13:59.409 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:13:59.409 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:14:00.038 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:14:00.063 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:14:00.121 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:14:00.153 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:14:00.206 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:14:00.249 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:14:00.279 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:14:00.307 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:14:00.340 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:14:00.374 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:14:00.400 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:14:00.402 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:14:00.402 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:14:00.402 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:14:00.402 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:14:00.402 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:14:00.402 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:14:00.403 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:14:00.403 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:14:00.403 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:14:00.403 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:14:00.403 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:14:00.403 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:14:00.403 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:14:00.982 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:14:00.982 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:14:02.456 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:14:02.565 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:14:02.983 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:14:03.198 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:14:05.312 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:14:05.344 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:14:05.357 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:14:05.387 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:14:05.415 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:14:05.441 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:14:05.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:14:05.533 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:14:05.541 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:14:05.541 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:14:05.541 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:14:05.541 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:14:05.542 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:14:05.573 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:14:05.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:14:05.632 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:14:05.658 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:14:05.660 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:14:05.660 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:14:05.660 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:14:05.660 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:14:05.660 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:14:05.660 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:14:05.661 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:14:05.661 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:14:05.661 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:14:05.661 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:14:05.661 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:14:05.661 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:14:05.661 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:14:06.379 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:14:06.569 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:14:06.569 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:14:08.405 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:14:08.546 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:14:08.570 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:14:09.401 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:14:09.402 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:14:10.912 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:14:11.115 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:14:11.885 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:14:11.975 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:14:12.014 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:14:12.048 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:14:12.083 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:14:12.115 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:14:12.142 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:14:12.169 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:14:12.196 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:14:12.223 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:14:12.249 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:14:12.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:14:12.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:14:12.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:14:12.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:14:12.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:14:12.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:14:12.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:14:12.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:14:12.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:14:12.253 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:14:12.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:14:12.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:14:12.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:14:12.768 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:14:12.768 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:14:13.053 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:14:13.053 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:14:14.401 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:14:14.875 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:14:15.054 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:14:15.313 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:14:15.314 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:14:16.943 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:14:17.349 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:14:17.375 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:14:17.427 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:14:17.452 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:14:17.479 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:14:17.505 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:14:17.530 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:14:17.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:14:17.584 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:14:17.609 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:14:17.635 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:14:17.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:14:17.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:14:17.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:14:17.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:14:17.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:14:17.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:14:17.638 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:14:17.638 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:14:17.638 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:14:17.638 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:14:17.638 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:14:17.638 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:14:17.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:14:17.667 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:14:18.244 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:14:18.244 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:14:18.792 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:14:18.792 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:14:20.443 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:14:20.793 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:14:20.902 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:14:21.351 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:14:21.351 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:14:22.956 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:14:23.310 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:14:23.335 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:14:23.369 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:14:23.381 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:14:23.408 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:14:23.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:14:23.460 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:14:23.486 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:14:23.509 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:14:23.521 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:14:23.548 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:14:23.574 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:14:23.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:14:23.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:14:23.578 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:14:23.578 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:14:23.578 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:14:23.578 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:14:23.578 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:14:23.578 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:14:23.578 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:14:23.579 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:14:23.579 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:14:23.579 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:14:23.579 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:14:24.216 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:14:24.216 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:14:24.381 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:14:24.382 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:14:26.217 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:14:26.567 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:14:27.170 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:14:27.170 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:14:28.921 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:14:28.945 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:14:28.977 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:14:29.008 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:14:29.039 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:14:29.070 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:14:29.099 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:14:29.128 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:14:29.155 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:14:29.182 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:14:29.209 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:14:29.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:14:29.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:14:29.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:14:29.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:14:29.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:14:29.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:14:29.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:14:29.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:14:29.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:14:29.213 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:14:29.213 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:14:29.213 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:14:29.213 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:14:29.426 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:14:29.762 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:14:29.763 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:14:30.098 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:14:30.099 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:14:31.764 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:14:32.028 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:14:32.761 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:14:32.761 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:14:33.874 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:14:33.896 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:14:33.919 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:14:33.944 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:14:33.971 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:14:33.997 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:14:34.023 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:14:34.072 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:14:34.124 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:14:34.168 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:14:34.215 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:14:34.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:14:34.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:14:34.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:14:34.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:14:34.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:14:34.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:14:34.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:14:34.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:14:34.220 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:14:34.220 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:14:34.220 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:14:34.220 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:14:34.221 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:14:34.837 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:14:34.838 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:14:35.121 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:14:35.646 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:14:35.647 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:14:36.562 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:14:36.844 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:14:38.142 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:14:38.776 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:14:38.776 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:14:39.293 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:14:39.301 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:14:39.327 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:14:39.352 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:14:39.375 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:14:39.402 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:14:39.428 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:14:39.460 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:14:39.481 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:14:39.508 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:14:39.536 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:14:39.573 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:14:39.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:14:39.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:14:39.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:14:39.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:14:39.574 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:14:39.574 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:14:39.574 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:14:39.574 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:14:39.574 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:14:39.574 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:14:39.574 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:14:39.575 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:14:39.575 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:14:40.294 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:14:40.295 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:14:41.399 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:14:41.867 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:14:41.868 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:14:42.296 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:14:42.588 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:14:44.337 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:14:44.818 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:14:44.855 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:14:44.856 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:14:44.864 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:14:44.892 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:14:44.923 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:14:44.955 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:14:44.983 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:14:45.009 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:14:45.035 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:14:45.061 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:14:45.088 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:14:45.122 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:14:45.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:14:45.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:14:45.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:14:45.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:14:45.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:14:45.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:14:45.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:14:45.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:14:45.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:14:45.126 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:14:45.126 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:14:45.126 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:14:45.126 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:14:45.617 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:14:45.708 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:14:45.709 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:14:47.209 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:14:47.709 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:14:47.736 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:14:47.736 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:14:48.425 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:14:50.329 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:14:50.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:14:50.403 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:14:50.426 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:14:50.457 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:14:50.493 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:14:50.525 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:14:50.530 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:14:50.554 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:14:50.581 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:14:50.624 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:14:50.663 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:14:50.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:14:50.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:14:50.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:14:50.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:14:50.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:14:50.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:14:50.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:14:50.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:14:50.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:14:50.667 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:14:50.667 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:14:50.667 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:14:50.667 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:14:50.985 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:14:50.986 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:14:51.056 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:14:51.333 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:14:51.334 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:14:53.286 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:14:53.334 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:14:53.992 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:14:53.992 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:14:54.425 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:14:55.959 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:14:55.985 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:14:56.009 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:14:56.044 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:14:56.070 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:14:56.096 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:14:56.122 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:14:56.149 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:14:56.176 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:14:56.203 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:14:56.232 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:14:56.234 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:14:56.235 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:14:56.235 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:14:56.235 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:14:56.235 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:14:56.235 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:14:56.235 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:14:56.235 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:14:56.236 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:14:56.236 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:14:56.236 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:14:56.236 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:14:56.236 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:14:56.705 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:14:56.794 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:14:56.794 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:14:57.140 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:14:57.141 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:14:57.209 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:14:58.796 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:15:00.042 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:15:00.454 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:15:00.892 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:15:00.893 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:15:01.522 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:15:01.548 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:15:01.572 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:15:01.602 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:15:01.634 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:15:01.662 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:15:01.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:15:01.721 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:15:01.747 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:15:01.774 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:15:01.800 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:15:01.803 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:15:01.803 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:15:01.803 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:15:01.803 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:15:01.803 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:15:01.804 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:15:01.804 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:15:01.804 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:15:01.804 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:15:01.804 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:15:01.804 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:15:01.804 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:15:01.805 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:15:02.447 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:15:02.447 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:15:03.026 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:15:03.218 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:15:03.545 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:15:03.546 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:15:04.448 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:15:05.894 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:15:06.562 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:15:06.648 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:15:06.648 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:15:07.493 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:15:08.350 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:15:08.396 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:15:08.434 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:15:08.471 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:15:08.506 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:15:08.546 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:15:08.586 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:15:08.629 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:15:08.646 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:15:08.673 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:15:08.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:15:08.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:15:08.676 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:15:08.676 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:15:08.676 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:15:08.676 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:15:08.677 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:15:08.677 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:15:08.677 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:15:08.677 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:15:08.678 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:15:08.678 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:15:08.678 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:15:08.782 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:15:09.235 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:15:09.236 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:15:09.273 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:15:09.405 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:15:09.406 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:15:11.237 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:15:11.682 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:15:12.090 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:15:12.413 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:15:14.265 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:15:14.287 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:15:14.311 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:15:14.335 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:15:14.361 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:15:14.387 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:15:14.413 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:15:14.440 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:15:14.466 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:15:14.493 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:15:14.533 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:15:14.535 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:15:14.535 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:15:14.535 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:15:14.535 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:15:14.536 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:15:14.536 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:15:14.536 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:15:14.536 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:15:14.536 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:15:14.536 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:15:14.536 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:15:14.537 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:15:14.537 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:15:14.760 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:15:14.760 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:15:14.760 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:15:14.761 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:15:14.761 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:15:14.851 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:15:15.104 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:15:15.105 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:15:15.438 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:15:17.106 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:15:17.699 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:15:17.699 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:15:17.699 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:15:17.700 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:15:17.700 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:15:18.332 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:15:18.440 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:15:20.089 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:15:20.127 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:15:20.151 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:15:20.191 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:15:20.226 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:15:20.253 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:15:20.280 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:15:20.306 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:15:20.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:15:20.360 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:15:20.388 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:15:20.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:15:20.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:15:20.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:15:20.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:15:20.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:15:20.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:15:20.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:15:20.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:15:20.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:15:20.391 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:15:20.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:15:20.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:15:20.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:15:20.870 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:15:20.882 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:15:20.882 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:15:21.226 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:15:21.316 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:15:22.884 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:15:23.575 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:15:24.167 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:15:24.472 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:15:25.923 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:15:25.954 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:15:25.990 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:15:26.023 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:15:26.054 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:15:26.083 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:15:26.109 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:15:26.136 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:15:26.175 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:15:26.211 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:15:26.245 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:15:26.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:15:26.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:15:26.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:15:26.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:15:26.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:15:26.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:15:26.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:15:26.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:15:26.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:15:26.251 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:15:26.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:15:26.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:15:26.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:15:26.732 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:15:26.733 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:15:26.791 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:15:26.792 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:15:26.792 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:15:26.792 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:15:26.792 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:15:27.221 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:15:27.282 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:15:28.734 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:15:29.949 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:15:29.949 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:15:29.949 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:15:29.949 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:15:30.566 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:15:30.735 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:15:31.607 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:15:32.459 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:15:32.503 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:15:32.541 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:15:32.599 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:15:32.646 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:15:32.682 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:15:32.717 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:15:32.776 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:15:32.826 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:15:32.858 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:15:32.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:15:32.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:15:32.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:15:32.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:15:32.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:15:32.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:15:32.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:15:32.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:15:32.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:15:32.870 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:15:32.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:15:32.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:15:32.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:15:32.900 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:15:33.191 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:15:33.423 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:15:33.424 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:15:33.569 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:15:35.425 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:15:35.886 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:15:36.574 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:15:36.589 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:15:38.121 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:15:38.983 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:15:39.026 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:15:39.061 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:15:39.095 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:15:39.095 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:15:39.095 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:15:39.095 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:15:39.096 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:15:39.129 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:15:39.161 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:15:39.191 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:15:39.219 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:15:39.253 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:15:39.275 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:15:39.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:15:39.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:15:39.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:15:39.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:15:39.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:15:39.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:15:39.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:15:39.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:15:39.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:15:39.278 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:15:39.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:15:39.279 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:15:39.279 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:15:39.443 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:15:39.578 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:15:40.011 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:15:40.011 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:15:42.006 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:15:42.007 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:15:42.007 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:15:42.007 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:15:42.007 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:15:42.012 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:15:42.464 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:15:42.495 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:15:44.854 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:15:44.877 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:15:44.904 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:15:44.924 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:15:44.950 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:15:44.981 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:15:45.004 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:15:45.032 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:15:45.059 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:15:45.089 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:15:45.101 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:15:45.107 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:15:45.113 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:15:45.114 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:15:45.114 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:15:45.114 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:15:45.114 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:15:45.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:15:45.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:15:45.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:15:45.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:15:45.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:15:45.115 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:15:45.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:15:45.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:15:45.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:15:45.804 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:15:46.084 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:15:46.085 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:15:48.086 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:15:48.139 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:15:48.456 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:15:48.663 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:15:48.669 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:15:50.987 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:15:51.166 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:15:51.278 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:15:51.840 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:15:51.892 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:15:51.894 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:15:51.894 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:15:51.932 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:15:51.966 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:15:51.997 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:15:52.028 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:15:52.061 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:15:52.088 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:15:52.132 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:15:52.175 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:15:52.176 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:15:52.177 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:15:52.177 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:15:52.177 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:15:52.177 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:15:52.177 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:15:52.178 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:15:52.178 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:15:52.178 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:15:52.178 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:15:52.179 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:15:52.179 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:15:52.179 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:15:52.763 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:15:52.763 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:15:54.281 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:15:54.457 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:15:54.765 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:15:54.814 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:15:54.815 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:15:57.197 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:15:57.283 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:15:57.317 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:15:57.350 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:15:57.379 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:15:57.406 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:15:57.434 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:15:57.465 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:15:57.492 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:15:57.518 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:15:57.532 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:15:57.545 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:15:57.571 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:15:57.572 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:15:57.572 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:15:57.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:15:57.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:15:57.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:15:57.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:15:57.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:15:57.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:15:57.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:15:57.573 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:15:57.574 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:15:57.574 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:15:57.574 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:15:58.461 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:15:58.461 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:15:58.467 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:15:58.467 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:16:00.469 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:16:00.489 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:16:00.577 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:16:01.404 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:16:01.405 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:16:03.132 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:16:03.218 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:16:03.249 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:16:03.274 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:16:03.297 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:16:03.323 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:16:03.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:16:03.391 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:16:03.417 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:16:03.451 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:16:03.481 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:16:03.509 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:16:03.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:16:03.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:16:03.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:16:03.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:16:03.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:16:03.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:16:03.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:16:03.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:16:03.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:16:03.512 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:16:03.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:16:03.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:16:03.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:16:03.744 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:16:04.229 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:16:04.229 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:16:04.707 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:16:04.707 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:16:06.230 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:16:06.573 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:16:06.619 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:16:07.475 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:16:07.475 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:16:08.830 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:16:08.854 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:16:08.879 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:16:08.904 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:16:08.931 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:16:08.957 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:16:08.983 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:16:09.010 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:16:09.036 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:16:09.066 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:16:09.094 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:16:09.095 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:16:09.095 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:16:09.095 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:16:09.095 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:16:09.095 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:16:09.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:16:09.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:16:09.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:16:09.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:16:09.096 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:16:09.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:16:09.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:16:09.097 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:16:09.680 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:16:09.680 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:16:09.758 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:16:09.947 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:16:10.572 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:16:10.572 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:16:11.681 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:16:12.493 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:16:13.347 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:16:14.076 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:16:14.077 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:16:14.469 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:16:14.495 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:16:14.520 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:16:14.543 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:16:14.569 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:16:14.619 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:16:14.659 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:16:14.699 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:16:14.729 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:16:14.758 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:16:14.785 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:16:14.788 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:16:14.788 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:16:14.788 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:16:14.788 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:16:14.789 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:16:14.789 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:16:14.789 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:16:14.789 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:16:14.789 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:16:14.789 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:16:14.789 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:16:14.790 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:16:14.790 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:16:15.574 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:16:15.625 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:16:15.626 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:16:16.489 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:16:17.374 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:16:17.375 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:16:17.626 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:16:18.496 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:16:20.092 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:16:20.116 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:16:20.139 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:16:20.163 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:16:20.190 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:16:20.216 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:16:20.242 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:16:20.268 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:16:20.294 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:16:20.302 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:16:20.320 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:16:20.347 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:16:20.348 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:16:20.349 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:16:20.349 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:16:20.349 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:16:20.349 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:16:20.349 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:16:20.349 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:16:20.349 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:16:20.350 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:16:20.350 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:16:20.350 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:16:20.350 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:16:20.350 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:16:20.917 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:16:20.918 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:16:21.067 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:16:21.068 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:16:21.498 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:16:23.069 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:16:23.153 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:16:23.789 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:16:23.789 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:16:24.472 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:16:26.177 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:16:26.204 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:16:26.229 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:16:26.254 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:16:26.279 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:16:26.305 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:16:26.332 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:16:26.358 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:16:26.358 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:16:26.384 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:16:26.412 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:16:26.439 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:16:26.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:16:26.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:16:26.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:16:26.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:16:26.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:16:26.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:16:26.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:16:26.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:16:26.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:16:26.442 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:16:26.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:16:26.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:16:26.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:16:27.002 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:16:27.002 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:16:27.137 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:16:27.138 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:16:27.429 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:16:29.138 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:16:29.168 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:16:29.784 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:16:29.784 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:16:30.591 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:16:32.184 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:16:32.206 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:16:32.227 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:16:32.230 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:16:32.254 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:16:32.281 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:16:32.307 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:16:32.334 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:16:32.360 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:16:32.396 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:16:32.413 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:16:32.439 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:16:32.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:16:32.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:16:32.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:16:32.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:16:32.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:16:32.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:16:32.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:16:32.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:16:32.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:16:32.442 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:16:32.443 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:16:32.443 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:16:32.443 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:16:32.746 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:16:32.747 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:16:33.076 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:16:33.076 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:16:33.398 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:16:34.978 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:16:35.078 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:16:35.589 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:16:35.590 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:16:36.592 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:16:37.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:16:37.930 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:16:37.954 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:16:37.978 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:16:38.006 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:16:38.032 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:16:38.058 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:16:38.084 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:16:38.111 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:16:38.137 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:16:38.157 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:16:38.163 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:16:38.164 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:16:38.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:16:38.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:16:38.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:16:38.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:16:38.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:16:38.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:16:38.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:16:38.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:16:38.166 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:16:38.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:16:38.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:16:38.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:16:38.828 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:16:38.828 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:16:38.979 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:16:38.979 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:16:39.454 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:16:40.980 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:16:41.084 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:16:41.852 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:16:41.853 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:16:42.506 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:16:44.067 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:16:44.095 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:16:44.123 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:16:44.147 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:16:44.173 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:16:44.199 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:16:44.226 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:16:44.252 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:16:44.278 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:16:44.297 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:16:44.305 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:16:44.331 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:16:44.332 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:16:44.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:16:44.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:16:44.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:16:44.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:16:44.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:16:44.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:16:44.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:16:44.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:16:44.334 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:16:44.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:16:44.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:16:44.334 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:16:44.863 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:16:44.863 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:16:45.027 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:16:45.027 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:16:45.364 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:16:47.028 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:16:47.132 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:16:47.710 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:16:47.711 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:16:48.497 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:16:50.000 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:16:50.026 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:16:50.049 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:16:50.073 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:16:50.104 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:16:50.130 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:16:50.157 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:16:50.183 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:16:50.209 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:16:50.236 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:16:50.253 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:16:50.262 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:16:50.264 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:16:50.264 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:16:50.264 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:16:50.264 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:16:50.264 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:16:50.265 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:16:50.265 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:16:50.265 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:16:50.265 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:16:50.265 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:16:50.265 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:16:50.266 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:16:50.266 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:16:50.907 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:16:50.908 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:16:51.088 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:16:51.088 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:16:51.295 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:16:53.090 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:16:53.264 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:16:53.264 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:16:53.264 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:16:53.264 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:16:53.935 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:16:54.496 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:16:56.141 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:16:56.179 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:16:56.211 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:16:56.238 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:16:56.271 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:16:56.302 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:16:56.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:16:56.360 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:16:56.386 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:16:56.413 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:16:56.439 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:16:56.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:16:56.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:16:56.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:16:56.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:16:56.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:16:56.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:16:56.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:16:56.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:16:56.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:16:56.442 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:16:56.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:16:56.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:16:56.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:16:56.457 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:16:57.052 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:16:57.052 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:16:57.219 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:16:57.254 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:16:59.053 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:16:59.398 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:17:00.094 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:17:00.527 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:17:02.238 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:17:02.524 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:17:02.524 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:17:02.524 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:17:02.525 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:17:03.040 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:17:03.090 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:17:03.130 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:17:03.161 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:17:03.189 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:17:03.217 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:17:03.244 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:17:03.270 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:17:03.297 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:17:03.324 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:17:03.351 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:17:03.352 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:17:03.352 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:17:03.353 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:17:03.353 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:17:03.353 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:17:03.353 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:17:03.353 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:17:03.353 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:17:03.354 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:17:03.354 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:17:03.354 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:17:03.354 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:17:03.354 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:17:03.401 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:17:04.033 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:17:04.033 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:17:05.643 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:17:05.643 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:17:05.643 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:17:05.644 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:17:05.644 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:17:06.035 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:17:06.134 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:17:06.579 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:17:08.538 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:17:08.563 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:17:08.587 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:17:08.610 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:17:08.637 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:17:08.663 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:17:08.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:17:08.718 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:17:08.753 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:17:08.780 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:17:08.780 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:17:08.807 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:17:08.808 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:17:08.808 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:17:08.808 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:17:08.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:17:08.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:17:08.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:17:08.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:17:08.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:17:08.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:17:08.809 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:17:08.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:17:08.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:17:08.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:17:09.291 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:17:09.363 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:17:09.544 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:17:09.544 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:17:11.545 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:17:11.791 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:17:12.390 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:17:14.180 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:17:14.753 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:17:15.030 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:17:15.080 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:17:15.129 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:17:15.172 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:17:15.205 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:17:15.239 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:17:15.272 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:17:15.298 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:17:15.339 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:17:15.374 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:17:15.404 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:17:15.407 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:17:15.407 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:17:15.407 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:17:15.407 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:17:15.408 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:17:15.408 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:17:15.408 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:17:15.408 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:17:15.408 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:17:15.408 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:17:15.409 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:17:15.409 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:17:15.409 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:17:15.885 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:17:15.885 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:17:17.539 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:17:17.539 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:17:17.539 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:17:17.539 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:17:17.540 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:17:17.886 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:17:18.039 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:17:20.474 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:17:20.712 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:17:20.712 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:17:20.712 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:17:20.712 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:17:21.255 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:17:21.331 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:17:21.382 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:17:21.419 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:17:21.454 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:17:21.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:17:21.513 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:17:21.552 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:17:21.579 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:17:21.605 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:17:21.631 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:17:21.633 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:17:21.633 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:17:21.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:17:21.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:17:21.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:17:21.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:17:21.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:17:21.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:17:21.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:17:21.635 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:17:21.635 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:17:21.635 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:17:21.635 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:17:22.242 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:17:22.242 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:17:22.503 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:17:23.525 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:17:24.029 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:17:24.243 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:17:25.128 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:17:27.122 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:17:27.237 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:17:27.734 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:17:28.097 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:17:28.134 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:17:28.171 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:17:28.226 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:17:28.275 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:17:28.316 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:17:28.357 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:17:28.404 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:17:28.434 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:17:28.465 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:17:28.467 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:17:28.467 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:17:28.467 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:17:28.467 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:17:28.468 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:17:28.468 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:17:28.468 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:17:28.468 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:17:28.468 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:17:28.468 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:17:28.469 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:17:28.469 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:17:28.469 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:17:28.509 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:17:29.450 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:17:29.451 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:17:29.995 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:17:30.515 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:17:30.516 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:17:31.368 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:17:31.451 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:17:33.358 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:17:34.046 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:17:34.047 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:17:34.729 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:17:35.512 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:17:35.581 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:17:35.629 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:17:35.663 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:17:35.699 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:17:35.726 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:17:35.753 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:17:35.783 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:17:35.811 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:17:35.840 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:17:35.867 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:17:35.868 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:17:35.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:17:35.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:17:35.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:17:35.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:17:35.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:17:35.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:17:35.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:17:35.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:17:35.870 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:17:35.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:17:35.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:17:35.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:17:36.245 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:17:36.536 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:17:36.537 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:17:36.857 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:17:36.857 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:17:38.381 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:17:38.538 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:17:38.785 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:17:39.639 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:17:39.639 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:17:41.536 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:17:41.900 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:17:41.935 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:17:41.966 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:17:41.995 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:17:42.024 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:17:42.070 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:17:42.106 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:17:42.140 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:17:42.169 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:17:42.196 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:17:42.223 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:17:42.225 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:17:42.225 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:17:42.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:17:42.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:17:42.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:17:42.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:17:42.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:17:42.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:17:42.227 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:17:42.227 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:17:42.227 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:17:42.227 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:17:42.227 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:17:42.582 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:17:42.702 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:17:42.703 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:17:43.085 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:17:43.085 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:17:44.242 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:17:44.703 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:17:44.998 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:17:45.699 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:17:45.699 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:17:47.504 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:17:48.165 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:17:48.195 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:17:48.232 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:17:48.264 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:17:48.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:17:48.323 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:17:48.350 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:17:48.376 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:17:48.424 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:17:48.471 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:17:48.508 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:17:48.510 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:17:48.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:17:48.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:17:48.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:17:48.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:17:48.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:17:48.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:17:48.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:17:48.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:17:48.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:17:48.513 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:17:48.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:17:48.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:17:48.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:17:49.153 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:17:49.154 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:17:49.325 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:17:49.326 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:17:50.252 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:17:51.326 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:17:51.372 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:17:52.144 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:17:52.145 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:17:53.525 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:17:54.571 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:17:54.617 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:17:55.088 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:17:55.089 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:17:55.422 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:17:55.470 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:17:55.512 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:17:55.546 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:17:55.576 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:17:55.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:17:55.633 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:17:55.685 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:17:55.727 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:17:55.762 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:17:55.764 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:17:55.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:17:55.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:17:55.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:17:55.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:17:55.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:17:55.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:17:55.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:17:55.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:17:55.766 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:17:55.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:17:55.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:17:55.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:17:56.300 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:17:56.307 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:17:56.307 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:17:57.475 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:17:58.155 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:17:58.155 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:17:58.308 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:17:59.541 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:18:00.799 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:18:01.332 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:18:01.332 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:18:01.415 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:18:01.449 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:18:01.477 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:18:01.502 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:18:01.529 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:18:01.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:18:01.586 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:18:01.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:18:01.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:18:01.666 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:18:01.692 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:18:01.694 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:18:01.695 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:18:01.695 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:18:01.695 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:18:01.695 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:18:01.695 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:18:01.695 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:18:01.696 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:18:01.696 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:18:01.696 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:18:01.696 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:18:01.696 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:18:01.696 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:18:02.378 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:18:02.391 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:18:02.392 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:18:03.577 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:18:04.369 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:18:04.370 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:18:04.393 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:18:05.583 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:18:06.953 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:18:06.984 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:18:07.008 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:18:07.043 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:18:07.077 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:18:07.116 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:18:07.145 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:18:07.171 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:18:07.184 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:18:07.198 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:18:07.224 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:18:07.251 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:18:07.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:18:07.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:18:07.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:18:07.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:18:07.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:18:07.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:18:07.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:18:07.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:18:07.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:18:07.254 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:18:07.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:18:07.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:18:07.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:18:07.819 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:18:07.819 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:18:07.982 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:18:07.982 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:18:08.267 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:18:09.983 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:18:10.019 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:18:10.630 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:18:10.631 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:18:11.134 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:18:12.955 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:18:12.989 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:18:13.017 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:18:13.044 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:18:13.074 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:18:13.107 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:18:13.135 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:18:13.161 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:18:13.169 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:18:13.188 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:18:13.215 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:18:13.243 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:18:13.245 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:18:13.245 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:18:13.245 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:18:13.245 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:18:13.245 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:18:13.245 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:18:13.246 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:18:13.246 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:18:13.246 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:18:13.246 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:18:13.246 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:18:13.246 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:18:13.833 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:18:13.833 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:18:13.982 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:18:13.983 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:18:14.183 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:18:15.984 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:18:16.119 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:18:16.668 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:18:16.668 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:18:17.535 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:18:19.149 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:18:19.149 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:18:19.171 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:18:19.195 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:18:19.218 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:18:19.247 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:18:19.275 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:18:19.303 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:18:19.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:18:19.366 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:18:19.418 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:18:19.461 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:18:19.464 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:18:19.464 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:18:19.464 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:18:19.464 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:18:19.464 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:18:19.465 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:18:19.465 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:18:19.465 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:18:19.465 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:18:19.465 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:18:19.465 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:18:19.466 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:18:19.466 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:18:19.635 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:18:19.635 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:18:19.979 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:18:19.979 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:18:20.553 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:18:21.981 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:18:22.019 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:18:22.768 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:18:22.769 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:18:23.574 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:18:25.361 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:18:25.509 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:18:25.544 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:18:25.573 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:18:25.599 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:18:25.627 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:18:25.654 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:18:25.681 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:18:25.708 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:18:25.735 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:18:25.762 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:18:25.788 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:18:25.790 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:18:25.790 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:18:25.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:18:25.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:18:25.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:18:25.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:18:25.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:18:25.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:18:25.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:18:25.792 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:18:25.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:18:25.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:18:25.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:18:25.850 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:18:25.851 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:18:26.246 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:18:26.247 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:18:26.765 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:18:28.166 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:18:28.248 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:18:28.705 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:18:28.706 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:18:29.557 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:18:31.057 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:18:31.081 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:18:31.107 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:18:31.131 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:18:31.158 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:18:31.184 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:18:31.213 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:18:31.239 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:18:31.265 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:18:31.292 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:18:31.312 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:18:31.318 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:18:31.320 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:18:31.320 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:18:31.320 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:18:31.320 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:18:31.320 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:18:31.321 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:18:31.321 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:18:31.321 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:18:31.321 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:18:31.321 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:18:31.321 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:18:31.321 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:18:31.322 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:18:31.947 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:18:31.947 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:18:32.122 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:18:32.122 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:18:32.382 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:18:34.123 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:18:34.387 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:18:34.388 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:18:34.388 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:18:34.388 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:18:35.009 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:18:35.562 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:18:37.069 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:18:37.409 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:18:37.922 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:18:37.930 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:18:37.969 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:18:38.010 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:18:38.042 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:18:38.091 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:18:38.130 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:18:38.160 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:18:38.193 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:18:38.220 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:18:38.246 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:18:38.248 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:18:38.248 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:18:38.248 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:18:38.248 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:18:38.248 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:18:38.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:18:38.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:18:38.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:18:38.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:18:38.249 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:18:38.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:18:38.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:18:38.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:18:38.257 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:18:38.873 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:18:38.874 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:18:40.442 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:18:40.875 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:18:40.924 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:18:41.573 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:18:43.524 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:18:43.560 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:18:43.595 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:18:43.622 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:18:43.663 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:18:43.710 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:18:43.736 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:18:43.737 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:18:43.737 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:18:43.737 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:18:43.746 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:18:43.781 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:18:43.810 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:18:43.840 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:18:43.874 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:18:43.876 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:18:43.876 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:18:43.876 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:18:43.876 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:18:43.877 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:18:43.877 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:18:43.877 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:18:43.877 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:18:43.877 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:18:43.877 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:18:43.877 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:18:43.878 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:18:43.878 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:18:44.175 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:18:44.241 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:18:44.560 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:18:44.560 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:18:46.561 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:18:46.639 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:18:46.639 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:18:46.639 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:18:46.639 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:18:46.639 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:18:46.902 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:18:47.106 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:18:49.480 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:18:49.502 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:18:49.526 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:18:49.550 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:18:49.561 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:18:49.577 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:18:49.603 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:18:49.604 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:18:49.629 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:18:49.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:18:49.697 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:18:49.725 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:18:49.751 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:18:49.753 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:18:49.753 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:18:49.753 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:18:49.753 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:18:49.753 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:18:49.753 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:18:49.754 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:18:49.754 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:18:49.754 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:18:49.754 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:18:49.754 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:18:49.754 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:18:49.754 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:18:50.072 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:18:50.520 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:18:50.521 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:18:52.515 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:18:52.521 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:18:53.003 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:18:53.106 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:18:55.268 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:18:55.293 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:18:55.324 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:18:55.353 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:18:55.382 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:18:55.409 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:18:55.436 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:18:55.463 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:18:55.478 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:18:55.478 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:18:55.478 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:18:55.479 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:18:55.479 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:18:55.490 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:18:55.516 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:18:55.542 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:18:55.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:18:55.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:18:55.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:18:55.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:18:55.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:18:55.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:18:55.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:18:55.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:18:55.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:18:55.545 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:18:55.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:18:55.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:18:55.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:18:55.882 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:18:56.100 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:18:56.249 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:18:56.249 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:18:58.249 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:18:58.372 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:18:58.373 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:18:58.373 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:18:58.373 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:18:58.373 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:18:58.630 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:18:58.909 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:19:01.237 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:19:01.437 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:19:01.551 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:19:01.551 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:19:01.551 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:19:01.551 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:19:02.040 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:19:02.090 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:19:02.128 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:19:02.163 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:19:02.192 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:19:02.222 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:19:02.248 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:19:02.276 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:19:02.303 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:19:02.329 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:19:02.356 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:19:02.357 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:19:02.357 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:19:02.357 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:19:02.358 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:19:02.358 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:19:02.358 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:19:02.358 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:19:02.358 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:19:02.358 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:19:02.358 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:19:02.358 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:19:02.359 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:19:02.359 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:19:03.415 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:19:03.415 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:19:04.592 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:19:04.674 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:19:05.237 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:19:05.417 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:19:07.378 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:19:07.548 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:19:07.813 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:19:07.842 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:19:07.871 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:19:07.898 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:19:07.925 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:19:07.951 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:19:07.978 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:19:08.004 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:19:08.030 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:19:08.075 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:19:08.085 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:19:08.125 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:19:08.128 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:19:08.128 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:19:08.128 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:19:08.128 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:19:08.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:19:08.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:19:08.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:19:08.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:19:08.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:19:08.130 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:19:08.130 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:19:08.130 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:19:08.130 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:19:08.931 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:19:08.931 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:19:10.571 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:19:10.669 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:19:10.932 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:19:11.384 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:19:11.385 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:19:13.204 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:19:13.628 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:19:13.657 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:19:13.681 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:19:13.703 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:19:13.704 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:19:13.730 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:19:13.756 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:19:13.782 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:19:13.809 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:19:13.835 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:19:13.861 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:19:13.886 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:19:13.888 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:19:13.888 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:19:13.888 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:19:13.888 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:19:13.888 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:19:13.889 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:19:13.889 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:19:13.889 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:19:13.889 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:19:13.889 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:19:13.889 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:19:13.889 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:19:13.890 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:19:14.574 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:19:14.575 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:19:14.850 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:19:14.850 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:19:16.583 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:19:16.671 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:19:16.851 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:19:17.329 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:19:17.329 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:19:19.153 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:19:19.287 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:19:19.316 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:19:19.342 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:19:19.367 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:19:19.395 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:19:19.421 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:19:19.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:19:19.474 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:19:19.498 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:19:19.524 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:19:19.547 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:19:19.558 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:19:19.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:19:19.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:19:19.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:19:19.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:19:19.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:19:19.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:19:19.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:19:19.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:19:19.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:19:19.561 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:19:19.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:19:19.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:19:19.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:19:20.298 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:19:20.298 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:19:20.434 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:19:20.436 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:19:22.300 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:19:22.471 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:19:22.976 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:19:23.201 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:19:23.201 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:19:24.719 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:19:24.743 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:19:24.765 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:19:24.788 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:19:24.813 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:19:24.864 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:19:24.906 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:19:24.945 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:19:24.979 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:19:25.012 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:19:25.049 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:19:25.051 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:19:25.051 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:19:25.051 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:19:25.051 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:19:25.051 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:19:25.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:19:25.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:19:25.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:19:25.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:19:25.052 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:19:25.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:19:25.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:19:25.053 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:19:25.594 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:19:25.701 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:19:25.702 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:19:25.963 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:19:26.553 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:19:26.553 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:19:27.702 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:19:28.614 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:19:28.940 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:19:30.092 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:19:30.092 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:19:30.526 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:19:30.560 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:19:30.593 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:19:30.618 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:19:30.645 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:19:30.673 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:19:30.699 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:19:30.727 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:19:30.752 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:19:30.789 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:19:30.821 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:19:30.822 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:19:30.822 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:19:30.822 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:19:30.823 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:19:30.823 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:19:30.823 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:19:30.823 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:19:30.823 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:19:30.823 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:19:30.823 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:19:30.824 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:19:30.824 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:19:30.824 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:19:31.538 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:19:31.539 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:19:31.820 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:19:32.293 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:19:32.900 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:19:32.900 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:19:33.540 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:19:34.593 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:19:35.549 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:19:36.270 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:19:36.270 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:19:36.367 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:19:37.121 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:19:37.222 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:19:37.245 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:19:37.270 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:19:37.297 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:19:37.324 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:19:37.350 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:19:37.377 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:19:37.402 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:19:37.426 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:19:37.450 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:19:37.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:19:37.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:19:37.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:19:37.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:19:37.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:19:37.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:19:37.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:19:37.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:19:37.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:19:37.454 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:19:37.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:19:37.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:19:37.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:19:38.244 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:19:38.244 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:19:38.806 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:19:39.382 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:19:39.383 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:19:40.245 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:19:40.586 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:19:41.718 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:19:42.390 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:19:42.390 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:19:42.943 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:19:42.967 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:19:42.991 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:19:43.016 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:19:43.043 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:19:43.070 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:19:43.096 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:19:43.123 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:19:43.160 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:19:43.195 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:19:43.225 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:19:43.227 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:19:43.227 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:19:43.227 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:19:43.228 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:19:43.228 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:19:43.228 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:19:43.228 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:19:43.228 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:19:43.228 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:19:43.228 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:19:43.229 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:19:43.229 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:19:43.229 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:19:43.375 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:19:44.071 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:19:44.071 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:19:44.700 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:19:45.334 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:19:45.335 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:19:46.072 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:19:46.619 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:19:47.924 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:19:48.390 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:19:48.390 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:19:48.940 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:19:48.965 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:19:48.990 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:19:49.014 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:19:49.040 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:19:49.067 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:19:49.093 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:19:49.118 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:19:49.145 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:19:49.172 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:19:49.207 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:19:49.207 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:19:49.207 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:19:49.207 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:19:49.207 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:19:49.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:19:49.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:19:49.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:19:49.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:19:49.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:19:49.208 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:19:49.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:19:49.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:19:49.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:19:49.305 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:19:50.127 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:19:50.127 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:19:50.745 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:19:51.475 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:19:51.475 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:19:52.105 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:19:52.128 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:19:53.800 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:19:54.466 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:19:54.467 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:19:54.687 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:19:54.987 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:19:55.011 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:19:55.036 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:19:55.059 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:19:55.086 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:19:55.112 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:19:55.138 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:19:55.165 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:19:55.192 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:19:55.221 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:19:55.247 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:19:55.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:19:55.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:19:55.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:19:55.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:19:55.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:19:55.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:19:55.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:19:55.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:19:55.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:19:55.250 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:19:55.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:19:55.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:19:55.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:19:56.366 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:19:56.366 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:19:57.077 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:19:57.553 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:19:57.676 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:19:57.677 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:19:58.367 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:19:59.853 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:20:00.007 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:20:00.450 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:20:00.450 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:20:00.884 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:20:01.736 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:20:01.788 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:20:01.824 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:20:01.862 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:20:01.891 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:20:01.918 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:20:01.945 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:20:01.971 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:20:02.014 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:20:02.050 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:20:02.053 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:20:02.053 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:20:02.053 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:20:02.054 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:20:02.054 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:20:02.054 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:20:02.054 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:20:02.054 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:20:02.054 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:20:02.054 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:20:02.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:20:02.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:20:02.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:20:02.741 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:20:02.742 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:20:03.419 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:20:03.603 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:20:04.047 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:20:04.047 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:20:04.742 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:20:06.430 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:20:06.471 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:20:07.218 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:20:07.218 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:20:07.381 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:20:07.403 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:20:07.429 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:20:07.455 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:20:07.481 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:20:07.507 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:20:07.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:20:07.560 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:20:07.587 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:20:07.620 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:20:07.645 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:20:07.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:20:07.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:20:07.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:20:07.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:20:07.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:20:07.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:20:07.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:20:07.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:20:07.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:20:07.648 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:20:07.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:20:07.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:20:07.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:20:08.801 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:20:08.801 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:20:09.725 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:20:10.402 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:20:10.403 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:20:10.636 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:20:10.803 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:20:13.275 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:20:13.406 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:20:13.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:20:13.939 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:20:13.940 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:20:14.753 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:20:14.782 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:20:14.812 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:20:14.840 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:20:14.866 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:20:14.892 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:20:14.918 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:20:14.945 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:20:14.971 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:20:14.997 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:20:14.999 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:20:14.999 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:20:14.999 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:20:14.999 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:20:15.000 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:20:15.000 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:20:15.000 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:20:15.000 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:20:15.000 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:20:15.000 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:20:15.000 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:20:15.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:20:15.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:20:15.906 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:20:15.907 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:20:16.831 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:20:16.958 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:20:17.426 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:20:17.908 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:20:19.847 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:20:20.162 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:20:20.475 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:20:20.504 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:20:20.532 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:20:20.558 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:20:20.588 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:20:20.612 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:20:20.639 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:20:20.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:20:20.694 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:20:20.721 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:20:20.759 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:20:20.760 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:20:20.760 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:20:20.760 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:20:20.760 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:20:20.760 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:20:20.760 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:20:20.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:20:20.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:20:20.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:20:20.761 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:20:20.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:20:20.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:20:20.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:20:21.144 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:20:21.785 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:20:21.785 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:20:23.748 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:20:23.748 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:20:23.748 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:20:23.748 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:20:23.748 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:20:23.786 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:20:24.024 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:20:24.538 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:20:26.859 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:20:26.880 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:20:26.919 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:20:26.954 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:20:26.988 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:20:27.016 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:20:27.043 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:20:27.048 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:20:27.071 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:20:27.097 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:20:27.127 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:20:27.154 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:20:27.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:20:27.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:20:27.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:20:27.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:20:27.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:20:27.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:20:27.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:20:27.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:20:27.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:20:27.157 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:20:27.158 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:20:27.158 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:20:27.158 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:20:27.397 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:20:28.095 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:20:28.095 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:20:28.469 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:20:30.096 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:20:30.610 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:20:30.690 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:20:31.949 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:20:32.728 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:20:32.751 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:20:32.774 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:20:32.798 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:20:32.825 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:20:32.851 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:20:32.877 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:20:32.904 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:20:32.932 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:20:32.959 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:20:32.985 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:20:32.986 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:20:32.987 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:20:32.987 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:20:32.987 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:20:32.987 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:20:32.987 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:20:32.987 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:20:32.987 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:20:32.988 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:20:32.988 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:20:32.988 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:20:32.988 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:20:32.988 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:20:33.590 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:20:33.914 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:20:33.914 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:20:34.877 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:20:34.878 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:20:34.878 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:20:34.878 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:20:34.878 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:20:35.719 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:20:35.915 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:20:36.633 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:20:38.566 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:20:38.566 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:20:38.566 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:20:38.567 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:20:38.676 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:20:38.699 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:20:38.747 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:20:38.747 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:20:38.771 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:20:38.797 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:20:38.823 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:20:38.850 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:20:38.876 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:20:38.929 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:20:38.970 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:20:38.973 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:20:38.973 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:20:38.973 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:20:38.973 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:20:38.973 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:20:38.973 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:20:38.973 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:20:38.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:20:38.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:20:38.974 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:20:38.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:20:38.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:20:38.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:20:39.138 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:20:39.566 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:20:39.566 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:20:39.768 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:20:41.567 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:20:41.781 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:20:42.725 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:20:43.631 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:20:44.966 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:20:44.989 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:20:45.013 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:20:45.036 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:20:45.063 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:20:45.091 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:20:45.119 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:20:45.146 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:20:45.172 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:20:45.200 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:20:45.227 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:20:45.230 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:20:45.230 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:20:45.230 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:20:45.230 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:20:45.231 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:20:45.231 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:20:45.231 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:20:45.231 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:20:45.231 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:20:45.231 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:20:45.231 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:20:45.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:20:45.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:20:45.760 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:20:45.761 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:20:45.761 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:20:45.761 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:20:45.761 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:20:45.769 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:20:45.769 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:20:46.331 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:20:46.629 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:20:47.770 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:20:48.840 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:20:48.840 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:20:48.840 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:20:48.840 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:20:49.721 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:20:50.646 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:20:52.530 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:20:52.530 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:20:52.530 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:20:52.530 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:20:52.530 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:20:53.141 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:20:53.382 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:20:55.684 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:20:55.684 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:20:55.685 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:20:55.685 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:20:55.685 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:20:56.303 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:20:56.647 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:20:58.468 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:20:59.055 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:20:59.056 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:20:59.391 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:21:01.360 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:21:01.539 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:21:01.564 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:21:01.591 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:21:01.616 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:21:01.678 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:21:01.678 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:21:01.696 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:21:01.722 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:21:01.749 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:21:01.775 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:21:01.818 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:21:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:21:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:21:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:21:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:21:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:21:01.819 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:21:01.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:21:01.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:21:01.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:21:01.820 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:21:01.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:21:01.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:21:01.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:21:02.071 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:21:02.071 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:21:02.470 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:21:02.604 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:21:02.604 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:21:04.414 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:21:04.605 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:21:04.965 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:21:04.965 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:21:05.189 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:21:07.831 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:21:07.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:21:07.875 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:21:07.875 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:21:07.899 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:21:07.928 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:21:07.955 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:21:07.981 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:21:08.010 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:21:08.042 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:21:08.068 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:21:08.130 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:21:08.130 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:21:08.130 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:21:08.130 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:21:08.130 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:21:08.131 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:21:08.131 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:21:08.131 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:21:08.131 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:21:08.131 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:21:08.131 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:21:08.131 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:21:08.132 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:21:08.132 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:21:08.545 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:21:08.545 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:21:08.658 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:21:09.068 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:21:09.069 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:21:10.982 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:21:11.069 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:21:11.700 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:21:11.741 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:21:11.741 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:21:14.154 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:21:14.192 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:21:14.220 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:21:14.243 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:21:14.270 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:21:14.297 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:21:14.348 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:21:14.393 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:21:14.435 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:21:14.468 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:21:14.508 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:21:14.509 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:21:14.510 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:21:14.510 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:21:14.510 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:21:14.510 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:21:14.510 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:21:14.510 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:21:14.510 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:21:14.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:21:14.511 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:21:14.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:21:14.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:21:14.511 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:21:15.247 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:21:15.391 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:21:15.391 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:21:15.659 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:21:15.890 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:21:15.891 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:21:17.392 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:21:18.062 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:21:18.386 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:21:18.670 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:21:18.670 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:21:20.673 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:21:20.700 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:21:20.722 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:21:20.746 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:21:20.772 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:21:20.799 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:21:20.826 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:21:20.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:21:20.881 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:21:20.914 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:21:20.941 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:21:20.943 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:21:20.943 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:21:20.943 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:21:20.943 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:21:20.944 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:21:20.944 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:21:20.944 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:21:20.944 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:21:20.944 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:21:20.944 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:21:20.944 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:21:20.945 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:21:20.945 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:21:21.646 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:21:21.646 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:21:21.679 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:21:22.208 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:21:22.775 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:21:22.775 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:21:23.646 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:21:24.346 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:21:24.885 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:21:26.065 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:21:26.065 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:21:26.773 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:21:26.798 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:21:26.820 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:21:26.844 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:21:26.870 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:21:26.903 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:21:26.936 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:21:26.963 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:21:27.002 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:21:27.051 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:21:27.074 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:21:27.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:21:27.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:21:27.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:21:27.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:21:27.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:21:27.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:21:27.077 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:21:27.077 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:21:27.077 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:21:27.077 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:21:27.077 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:21:27.077 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:21:27.078 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:21:27.639 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:21:28.142 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:21:28.142 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:21:29.034 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:21:29.654 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:21:29.654 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:21:30.143 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:21:30.495 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:21:32.434 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:21:33.345 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:21:33.346 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:21:33.414 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:21:34.338 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:21:34.339 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:21:34.339 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:21:34.360 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:21:34.386 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:21:34.416 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:21:34.442 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:21:34.495 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:21:34.545 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:21:34.587 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:21:34.590 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:21:34.590 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:21:34.590 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:21:34.590 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:21:34.590 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:21:34.591 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:21:34.591 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:21:34.591 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:21:34.591 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:21:34.591 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:21:34.591 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:21:34.591 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:21:34.592 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:21:34.675 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:21:35.336 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:21:35.336 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:21:35.992 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:21:36.611 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:21:36.611 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:21:37.337 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:21:37.628 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:21:39.345 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:21:40.530 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:21:40.531 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:21:40.552 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:21:40.574 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:21:40.599 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:21:40.624 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:21:40.658 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:21:40.688 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:21:40.717 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:21:40.757 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:21:40.785 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:21:40.814 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:21:40.845 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:21:40.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:21:40.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:21:40.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:21:40.847 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:21:40.847 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:21:40.847 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:21:40.847 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:21:40.847 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:21:40.847 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:21:40.847 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:21:40.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:21:40.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:21:40.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:21:41.642 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:21:41.643 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:21:41.668 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:21:43.054 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:21:43.613 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:21:43.614 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:21:43.644 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:21:44.532 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:21:46.197 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:21:46.937 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:21:46.961 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:21:46.991 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:21:47.015 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:21:47.040 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:21:47.070 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:21:47.098 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:21:47.122 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:21:47.123 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:21:47.134 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:21:47.168 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:21:47.196 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:21:47.224 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:21:47.225 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:21:47.225 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:21:47.225 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:21:47.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:21:47.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:21:47.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:21:47.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:21:47.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:21:47.226 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:21:47.226 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:21:47.227 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:21:47.227 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:21:47.227 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:21:48.009 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:21:48.009 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:21:48.985 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:21:49.656 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:21:50.010 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:21:50.210 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:21:50.211 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:21:51.626 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:21:52.451 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:21:52.475 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:21:52.501 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:21:52.529 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:21:52.555 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:21:52.582 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:21:52.621 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:21:52.647 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:21:52.676 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:21:52.704 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:21:52.734 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:21:52.735 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:21:52.735 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:21:52.735 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:21:52.735 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:21:52.736 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:21:52.736 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:21:52.736 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:21:52.736 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:21:52.736 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:21:52.736 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:21:52.736 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:21:52.737 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:21:52.737 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:21:52.914 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:21:53.931 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:21:53.931 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:21:54.197 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:21:54.197 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:21:55.108 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:21:55.932 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:21:56.787 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:21:57.383 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:21:57.383 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:21:57.961 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:21:58.693 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:21:58.717 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:21:58.741 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:21:58.769 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:21:58.792 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:21:58.819 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:21:58.845 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:21:58.875 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:21:58.901 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:21:58.933 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:21:58.959 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:21:58.961 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:21:58.961 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:21:58.962 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:21:58.962 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:21:58.962 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:21:58.962 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:21:58.962 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:21:58.962 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:21:58.962 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:21:58.962 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:21:58.963 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:21:58.963 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:21:58.963 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:21:59.831 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:21:59.832 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:22:00.409 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:22:01.000 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:22:01.001 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:22:01.679 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:22:01.833 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:22:03.325 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:22:04.201 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:22:04.202 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:22:04.667 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:22:04.837 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:22:04.865 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:22:04.892 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:22:04.917 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:22:04.945 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:22:04.976 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:22:05.002 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:22:05.028 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:22:05.055 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:22:05.093 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:22:05.111 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:22:05.112 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:22:05.113 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:22:05.113 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:22:05.113 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:22:05.113 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:22:05.113 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:22:05.113 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:22:05.113 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:22:05.114 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:22:05.114 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:22:05.114 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:22:05.114 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:22:05.114 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:22:06.116 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:22:06.116 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:22:07.295 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:22:07.907 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:22:07.907 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:22:08.117 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:22:08.726 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:22:10.690 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:22:10.913 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:22:10.937 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:22:10.966 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:22:10.987 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:22:11.013 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:22:11.039 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:22:11.064 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:22:11.090 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:22:11.117 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:22:11.152 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:22:11.182 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:22:11.184 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:22:11.184 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:22:11.184 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:22:11.184 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:22:11.184 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:22:11.184 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:22:11.185 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:22:11.185 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:22:11.185 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:22:11.185 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:22:11.185 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:22:11.185 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:22:11.185 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:22:11.365 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:22:11.759 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:22:11.760 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:22:11.893 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:22:13.760 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:22:13.866 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:22:13.866 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:22:13.866 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:22:13.866 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:22:13.866 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:22:14.693 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:22:14.710 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:22:16.558 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:22:16.582 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:22:16.608 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:22:16.633 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:22:16.664 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:22:16.691 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:22:16.717 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:22:16.746 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:22:16.772 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:22:16.801 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:22:16.827 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:22:16.831 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:22:16.831 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:22:16.832 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:22:16.832 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:22:16.832 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:22:16.832 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:22:16.832 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:22:16.833 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:22:16.835 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:22:16.835 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:22:16.835 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:22:16.835 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:22:16.835 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:22:17.293 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:22:17.384 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:22:17.384 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:22:17.669 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:22:17.789 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:22:19.385 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:22:20.185 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:22:20.688 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:22:20.735 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:22:22.375 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:22:22.408 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:22:22.438 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:22:22.466 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:22:22.494 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:22:22.525 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:22:22.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:22:22.587 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:22:22.617 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:22:22.645 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:22:22.672 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:22:22.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:22:22.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:22:22.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:22:22.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:22:22.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:22:22.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:22:22.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:22:22.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:22:22.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:22:22.675 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:22:22.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:22:22.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:22:22.676 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:22:23.197 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:22:23.197 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:22:23.197 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:22:23.198 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:22:23.198 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:22:23.257 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:22:23.257 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:22:23.473 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:22:23.603 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:22:25.258 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:22:25.784 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:22:25.784 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:22:25.784 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:22:25.784 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:22:25.785 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:22:26.463 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:22:26.714 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:22:28.610 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:22:28.633 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:22:28.671 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:22:28.706 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:22:28.739 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:22:28.770 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:22:28.800 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:22:28.828 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:22:28.855 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:22:28.882 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:22:28.907 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:22:28.910 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:22:28.910 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:22:28.910 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:22:28.911 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:22:28.911 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:22:28.911 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:22:28.911 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:22:28.911 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:22:28.911 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:22:28.911 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:22:28.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:22:28.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:22:28.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:22:28.960 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:22:29.495 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:22:29.502 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:22:29.503 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:22:29.651 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:22:31.504 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:22:32.062 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:22:32.102 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:22:32.857 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:22:33.738 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:22:34.591 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:22:34.642 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:22:34.682 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:22:34.718 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:22:34.752 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:22:34.781 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:22:34.808 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:22:34.835 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:22:34.863 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:22:34.889 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:22:34.890 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:22:34.891 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:22:34.891 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:22:34.891 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:22:34.891 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:22:34.891 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:22:34.891 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:22:34.891 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:22:34.892 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:22:34.892 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:22:34.892 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:22:34.892 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:22:34.892 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:22:34.939 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:22:35.058 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:22:35.058 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:22:35.059 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:22:35.059 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:22:35.059 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:22:35.434 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:22:35.434 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:22:35.958 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:22:37.435 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:22:38.265 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:22:38.265 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:22:38.265 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:22:38.265 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:22:38.696 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:22:38.849 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:22:40.571 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:22:40.599 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:22:40.626 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:22:40.652 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:22:40.677 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:22:40.707 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:22:40.733 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:22:40.759 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:22:40.785 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:22:40.813 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:22:40.841 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:22:40.843 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:22:40.843 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:22:40.843 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:22:40.843 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:22:40.843 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:22:40.843 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:22:40.844 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:22:40.844 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:22:40.844 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:22:40.844 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:22:40.844 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:22:40.844 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:22:40.845 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:22:41.397 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:22:41.397 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:22:41.485 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:22:41.632 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:22:42.035 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:22:43.398 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:22:44.250 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:22:44.712 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:22:44.842 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:22:46.287 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:22:46.317 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:22:46.347 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:22:46.375 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:22:46.404 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:22:46.432 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:22:46.459 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:22:46.486 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:22:46.513 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:22:46.540 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:22:46.584 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:22:46.586 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:22:46.586 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:22:46.587 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:22:46.587 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:22:46.587 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:22:46.587 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:22:46.587 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:22:46.587 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:22:46.587 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:22:46.588 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:22:46.588 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:22:46.588 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:22:46.588 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:22:47.211 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:22:47.211 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:22:47.541 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:22:47.569 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:22:48.082 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:22:48.082 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:22:49.212 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:22:50.520 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:22:50.664 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:22:51.440 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:22:51.440 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:22:52.150 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:22:53.005 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:22:53.052 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:22:53.091 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:22:53.122 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:22:53.154 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:22:53.191 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:22:53.226 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:22:53.257 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:22:53.298 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:22:53.312 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:22:53.314 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:22:53.314 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:22:53.315 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:22:53.315 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:22:53.315 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:22:53.315 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:22:53.315 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:22:53.315 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:22:53.315 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:22:53.316 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:22:53.316 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:22:53.316 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:22:53.316 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:22:53.393 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:22:53.706 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:22:53.973 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:22:53.974 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:22:54.178 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:22:54.178 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:22:55.975 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:22:56.509 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:22:56.669 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:22:57.316 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:22:57.317 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:22:59.034 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:22:59.066 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:22:59.100 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:22:59.124 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:22:59.159 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:22:59.187 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:22:59.215 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:22:59.243 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:22:59.270 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:22:59.297 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:22:59.323 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:22:59.325 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:22:59.325 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:22:59.326 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:22:59.326 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:22:59.326 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:22:59.326 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:22:59.326 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:22:59.327 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:22:59.327 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:22:59.327 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:22:59.327 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:22:59.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:22:59.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:22:59.828 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:22:59.846 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:22:59.847 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:22:59.864 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:23:00.482 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:23:00.482 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:23:01.847 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:23:02.707 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:23:03.082 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:23:04.169 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:23:04.170 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:23:04.566 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:23:05.416 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:23:05.468 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:23:05.511 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:23:05.561 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:23:05.589 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:23:05.617 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:23:05.643 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:23:05.670 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:23:05.697 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:23:05.725 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:23:05.727 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:23:05.727 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:23:05.727 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:23:05.727 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:23:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:23:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:23:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:23:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:23:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:23:05.728 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:23:05.729 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:23:05.729 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:23:05.729 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:23:05.906 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:23:06.206 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:23:06.350 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:23:06.351 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:23:06.879 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:23:06.880 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:23:08.351 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:23:08.751 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:23:09.238 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:23:09.935 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:23:09.935 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:23:11.391 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:23:11.432 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:23:11.467 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:23:11.495 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:23:11.504 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:23:11.523 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:23:11.550 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:23:11.576 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:23:11.602 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:23:11.630 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:23:11.657 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:23:11.684 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:23:11.695 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:23:11.695 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:23:11.695 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:23:11.695 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:23:11.695 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:23:11.696 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:23:11.696 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:23:11.696 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:23:11.696 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:23:11.696 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:23:11.696 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:23:11.696 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:23:11.697 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:23:12.028 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:23:12.469 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:23:12.469 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:23:12.939 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:23:12.939 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:23:14.470 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:23:14.723 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:23:14.998 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:23:15.841 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:23:15.841 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:23:17.092 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:23:17.115 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:23:17.140 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:23:17.164 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:23:17.191 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:23:17.217 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:23:17.248 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:23:17.270 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:23:17.301 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:23:17.329 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:23:17.364 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:23:17.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:23:17.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:23:17.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:23:17.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:23:17.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:23:17.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:23:17.366 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:23:17.366 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:23:17.366 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:23:17.366 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:23:17.366 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:23:17.366 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:23:17.366 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:23:17.429 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:23:17.905 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:23:17.905 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:23:18.159 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:23:18.705 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:23:18.706 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:23:19.906 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:23:20.723 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:23:21.481 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:23:22.220 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:23:22.221 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:23:22.665 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:23:23.334 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:23:23.529 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:23:23.560 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:23:23.630 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:23:23.685 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:23:23.711 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:23:23.746 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:23:23.777 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:23:23.806 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:23:23.856 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:23:23.896 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:23:23.899 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:23:23.899 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:23:23.899 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:23:23.899 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:23:23.899 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:23:23.899 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:23:23.900 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:23:23.900 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:23:23.900 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:23:23.900 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:23:23.900 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:23:23.901 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:23:23.901 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:23:24.515 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:23:24.872 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:23:24.872 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:23:25.157 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:23:25.157 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:23:26.752 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:23:26.873 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:23:27.714 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:23:28.322 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:23:28.323 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:23:29.519 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:23:29.573 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:23:30.427 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:23:30.468 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:23:30.495 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:23:30.524 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:23:30.551 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:23:30.577 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:23:30.603 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:23:30.629 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:23:30.638 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:23:30.658 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:23:30.686 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:23:30.687 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:23:30.687 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:23:30.687 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:23:30.688 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:23:30.688 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:23:30.688 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:23:30.688 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:23:30.689 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:23:30.689 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:23:30.689 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:23:30.689 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:23:30.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:23:30.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:23:31.485 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:23:31.485 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:23:31.731 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:23:31.731 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:23:33.590 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:23:33.732 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:23:34.077 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:23:34.077 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:23:36.166 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:23:36.188 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:23:36.211 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:23:36.233 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:23:36.234 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:23:36.261 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:23:36.287 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:23:36.315 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:23:36.343 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:23:36.369 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:23:36.424 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:23:36.461 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:23:36.463 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:23:36.464 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:23:36.464 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:23:36.464 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:23:36.464 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:23:36.464 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:23:36.464 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:23:36.464 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:23:36.466 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:23:36.466 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:23:36.466 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:23:36.466 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:23:36.466 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:23:36.747 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:23:36.748 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:23:37.113 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:23:37.113 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:23:38.786 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:23:39.114 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:23:39.294 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:23:39.295 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:23:41.624 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:23:41.923 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:23:42.477 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:23:42.491 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:23:42.492 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:23:42.526 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:23:42.564 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:23:42.599 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:23:42.628 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:23:42.656 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:23:42.681 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:23:42.708 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:23:42.718 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:23:42.756 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:23:42.784 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:23:42.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:23:42.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:23:42.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:23:42.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:23:42.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:23:42.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:23:42.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:23:42.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:23:42.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:23:42.787 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:23:42.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:23:42.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:23:42.788 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:23:43.523 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:23:43.524 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:23:44.798 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:23:45.297 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:23:45.298 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:23:45.524 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:23:45.773 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:23:48.205 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:23:48.481 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:23:48.502 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:23:48.532 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:23:48.561 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:23:48.589 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:23:48.608 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:23:48.634 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:23:48.678 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:23:48.724 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:23:48.758 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:23:48.789 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:23:48.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:23:48.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:23:48.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:23:48.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:23:48.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:23:48.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:23:48.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:23:48.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:23:48.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:23:48.792 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:23:48.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:23:48.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:23:48.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:23:48.909 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:23:48.909 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:23:49.281 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:23:49.281 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:23:49.736 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:23:51.242 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:23:51.242 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:23:51.242 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:23:51.242 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:23:51.282 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:23:51.786 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:23:52.578 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:23:53.952 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:23:53.974 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:23:53.998 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:23:54.024 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:23:54.049 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:23:54.076 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:23:54.102 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:23:54.128 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:23:54.157 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:23:54.186 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:23:54.215 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:23:54.217 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:23:54.217 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:23:54.217 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:23:54.217 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:23:54.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:23:54.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:23:54.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:23:54.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:23:54.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:23:54.218 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:23:54.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:23:54.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:23:54.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:23:54.584 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:23:55.101 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:23:55.102 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:23:55.365 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:23:56.761 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:23:57.103 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:23:57.503 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:23:58.091 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:23:59.486 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:23:59.512 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:23:59.552 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:23:59.579 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:23:59.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:23:59.636 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:23:59.669 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:23:59.698 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:23:59.724 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:23:59.750 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:23:59.801 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:23:59.851 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:23:59.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:23:59.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:23:59.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:23:59.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:23:59.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:23:59.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:23:59.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:23:59.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:23:59.853 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:23:59.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:23:59.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:23:59.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:24:00.711 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:24:00.711 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:24:01.097 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:24:01.098 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:24:01.098 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:24:01.098 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:24:01.737 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:24:02.575 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:24:02.712 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:24:04.134 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:24:04.134 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:24:04.134 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:24:04.134 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:24:04.134 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:24:04.821 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:24:05.441 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:24:05.463 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:24:05.499 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:24:05.524 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:24:05.554 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:24:05.581 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:24:05.610 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:24:05.636 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:24:05.664 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:24:05.708 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:24:05.735 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:24:05.761 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:24:05.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:24:05.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:24:05.763 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:24:05.763 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:24:05.763 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:24:05.763 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:24:05.763 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:24:05.763 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:24:05.763 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:24:05.764 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:24:05.764 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:24:05.764 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:24:05.764 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:24:06.497 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:24:06.497 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:24:07.645 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:24:08.172 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:24:08.498 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:24:08.743 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:24:10.949 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:24:11.210 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:24:11.241 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:24:11.274 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:24:11.305 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:24:11.335 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:24:11.364 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:24:11.392 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:24:11.419 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:24:11.449 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:24:11.465 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:24:11.498 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:24:11.520 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:24:11.539 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:24:11.539 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:24:11.539 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:24:11.539 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:24:11.540 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:24:11.540 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:24:11.540 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:24:11.540 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:24:11.540 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:24:11.540 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:24:11.540 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:24:11.541 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:24:11.541 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:24:11.541 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:24:11.958 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:24:11.959 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:24:13.961 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:24:14.261 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:24:14.261 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:24:14.261 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:24:14.261 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:24:14.262 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:24:14.760 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:24:14.888 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:24:16.729 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:24:16.766 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:24:16.802 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:24:16.833 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:24:16.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:24:16.892 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:24:16.919 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:24:16.946 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:24:16.974 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:24:17.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:24:17.028 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:24:17.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:24:17.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:24:17.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:24:17.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:24:17.030 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:24:17.030 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:24:17.030 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:24:17.030 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:24:17.030 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:24:17.030 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:24:17.030 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:24:17.031 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:24:17.031 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:24:17.312 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:24:17.566 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:24:17.567 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:24:17.570 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:24:18.158 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:24:19.568 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:24:20.761 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:24:20.807 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:24:21.829 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:24:22.265 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:24:22.287 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:24:22.310 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:24:22.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:24:22.358 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:24:22.387 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:24:22.413 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:24:22.440 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:24:22.484 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:24:22.519 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:24:22.551 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:24:22.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:24:22.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:24:22.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:24:22.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:24:22.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:24:22.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:24:22.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:24:22.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:24:22.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:24:22.554 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:24:22.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:24:22.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:24:22.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:24:23.324 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:24:23.324 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:24:23.585 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:24:24.078 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:24:24.079 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:24:24.079 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:24:24.079 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:24:24.079 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:24:24.648 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:24:25.325 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:24:26.805 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:24:27.129 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:24:27.130 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:24:27.130 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:24:27.130 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:24:27.130 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:24:27.963 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:24:28.234 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:24:29.086 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:24:29.140 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:24:29.181 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:24:29.217 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:24:29.250 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:24:29.285 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:24:29.315 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:24:29.373 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:24:29.415 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:24:29.451 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:24:29.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:24:29.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:24:29.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:24:29.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:24:29.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:24:29.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:24:29.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:24:29.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:24:29.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:24:29.455 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:24:29.455 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:24:29.455 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:24:29.455 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:24:29.534 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:24:30.021 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:24:30.108 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:24:30.109 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:24:30.701 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:24:30.701 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:24:32.110 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:24:33.100 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:24:33.106 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:24:34.247 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:24:34.248 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:24:34.665 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:24:34.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:24:34.713 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:24:34.736 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:24:34.763 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:24:34.789 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:24:34.816 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:24:34.841 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:24:34.870 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:24:34.898 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:24:34.936 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:24:34.936 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:24:34.936 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:24:34.937 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:24:34.937 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:24:34.937 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:24:34.937 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:24:34.937 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:24:34.937 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:24:34.937 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:24:34.938 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:24:34.938 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:24:34.938 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:24:34.938 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:24:35.643 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:24:35.644 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:24:35.990 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:24:36.395 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:24:37.107 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:24:37.108 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:24:37.644 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:24:38.763 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:24:39.619 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:24:40.129 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:24:40.130 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:24:40.725 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:24:41.576 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:24:41.590 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:24:41.632 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:24:41.663 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:24:41.691 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:24:41.719 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:24:41.751 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:24:41.779 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:24:41.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:24:41.834 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:24:41.863 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:24:41.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:24:41.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:24:41.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:24:41.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:24:41.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:24:41.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:24:41.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:24:42.069 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:24:42.801 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:24:42.802 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:24:42.967 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:24:42.968 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:24:44.783 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:24:44.968 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:24:45.037 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:24:45.693 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:24:45.694 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:24:47.527 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:24:47.631 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:24:47.656 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:24:47.681 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:24:47.716 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:24:47.743 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:24:47.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:24:47.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:24:47.834 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:24:47.863 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:24:47.866 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:24:47.895 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:24:47.921 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:24:47.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:24:47.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:24:47.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:24:47.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:24:47.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:24:47.924 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:24:47.924 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:24:47.924 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:24:47.924 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:24:47.924 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:24:47.924 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:24:47.924 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:24:47.925 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:24:48.512 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:24:48.513 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:24:49.131 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:24:49.131 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:24:50.710 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:24:51.133 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:24:51.231 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:24:51.766 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:24:51.767 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:24:53.278 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:24:53.703 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:24:53.725 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:24:53.749 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:24:53.775 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:24:53.800 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:24:53.827 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:24:53.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:24:53.879 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:24:53.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:24:53.934 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:24:53.961 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:24:53.962 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:24:53.962 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:24:53.963 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:24:53.963 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:24:53.963 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:24:53.963 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:24:53.964 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:24:53.964 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:24:53.964 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:24:53.964 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:24:53.964 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:24:53.964 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:24:53.964 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:24:54.154 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:24:54.865 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:24:54.865 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:24:55.307 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:24:55.308 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:24:56.697 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:24:56.866 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:24:57.555 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:24:58.140 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:24:58.141 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:24:59.316 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:24:59.594 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:24:59.618 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:24:59.664 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:24:59.687 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:24:59.713 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:24:59.739 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:24:59.767 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:24:59.793 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:24:59.820 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:24:59.848 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:24:59.878 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:24:59.880 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:24:59.880 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:24:59.880 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:24:59.881 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:24:59.881 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:24:59.881 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:24:59.881 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:24:59.881 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:24:59.881 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:24:59.882 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:24:59.882 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:24:59.882 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:24:59.882 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:25:00.592 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:25:00.593 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:25:00.957 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:25:01.537 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:25:01.538 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:25:02.163 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:25:02.593 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:25:03.928 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:25:04.647 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:25:04.647 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:25:04.768 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:25:05.046 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:25:05.897 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:25:05.945 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:25:05.985 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:25:06.023 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:25:06.057 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:25:06.090 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:25:06.135 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:25:06.176 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:25:06.207 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:25:06.237 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:25:06.239 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:25:06.239 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:25:06.239 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:25:06.240 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:25:06.240 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:25:06.240 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:25:06.240 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:25:06.240 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:25:06.240 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:25:06.241 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:25:06.241 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:25:06.241 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:25:06.241 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:25:06.994 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:25:06.994 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:25:07.711 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:25:08.194 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:25:08.291 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:25:08.292 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:25:08.996 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:25:10.676 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:25:10.691 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:25:11.456 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:25:11.457 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:25:11.628 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:25:11.652 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:25:11.676 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:25:11.700 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:25:11.726 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:25:11.761 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:25:11.787 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:25:11.824 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:25:11.839 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:25:11.866 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:25:11.895 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:25:11.896 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:25:11.896 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:25:11.896 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:25:11.897 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:25:11.897 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:25:11.897 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:25:11.897 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:25:11.897 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:25:11.897 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:25:11.897 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:25:11.898 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:25:11.898 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:25:11.898 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:25:12.811 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:25:12.811 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:25:13.981 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:25:14.513 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:25:14.513 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:25:14.796 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:25:14.812 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:25:17.097 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:25:17.464 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:25:17.595 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:25:17.595 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:25:17.646 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:25:18.502 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:25:18.544 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:25:18.589 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:25:18.626 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:25:18.661 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:25:18.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:25:18.725 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:25:18.752 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:25:18.779 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:25:18.806 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:25:18.807 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:25:18.807 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:25:18.808 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:25:18.808 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:25:18.808 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:25:18.808 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:25:18.808 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:25:18.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:25:18.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:25:18.809 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:25:18.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:25:18.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:25:18.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:25:19.652 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:25:19.652 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:25:20.315 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:25:20.798 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:25:20.811 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:25:20.812 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:25:21.653 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:25:23.160 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:25:23.280 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:25:23.656 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:25:23.656 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:25:24.595 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:25:24.617 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:25:24.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:25:24.664 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:25:24.690 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:25:24.719 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:25:24.747 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:25:24.773 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:25:24.800 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:25:24.830 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:25:24.869 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:25:24.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:25:24.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:25:24.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:25:24.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:25:24.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:25:24.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:25:24.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:25:24.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:25:24.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:25:24.872 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:25:24.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:25:24.873 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:25:24.873 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:25:25.829 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:25:25.829 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:25:26.495 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:25:26.858 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:25:27.089 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:25:27.090 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:25:27.830 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:25:29.407 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:25:29.450 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:25:29.903 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:25:29.904 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:25:30.513 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:25:30.537 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:25:30.559 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:25:30.583 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:25:30.609 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:25:30.635 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:25:30.662 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:25:30.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:25:30.722 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:25:30.747 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:25:30.774 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:25:30.775 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:25:30.776 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:25:30.776 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:25:30.776 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:25:30.776 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:25:30.776 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:25:30.776 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:25:30.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:25:30.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:25:30.777 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:25:30.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:25:30.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:25:30.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:25:31.754 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:25:31.755 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:25:32.710 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:25:32.756 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:25:33.533 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:25:33.533 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:25:33.755 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:25:35.366 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:25:35.789 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:25:36.262 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:25:36.295 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:25:36.324 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:25:36.353 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:25:36.382 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:25:36.409 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:25:36.441 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:25:36.463 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:25:36.489 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:25:36.494 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:25:36.520 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:25:36.559 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:25:36.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:25:36.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:25:36.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:25:36.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:25:36.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:25:36.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:25:36.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:25:36.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:25:36.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:25:36.560 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:25:36.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:25:36.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:25:36.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:25:37.106 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:25:37.109 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:25:38.802 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:25:39.063 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:25:39.063 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:25:39.063 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:25:39.064 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:25:39.064 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:25:39.110 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:25:39.723 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:25:41.364 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:25:41.736 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:25:41.758 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:25:41.783 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:25:41.808 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:25:41.837 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:25:41.864 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:25:41.891 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:25:41.921 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:25:41.949 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:25:41.984 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:25:42.012 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:25:42.013 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:25:42.013 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:25:42.013 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:25:42.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:25:42.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:25:42.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:25:42.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:25:42.015 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:25:42.015 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:25:42.015 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:25:42.015 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:25:42.015 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:25:42.016 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:25:42.110 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:25:42.761 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:25:42.761 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:25:43.273 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:25:44.762 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:25:44.802 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:25:45.446 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:25:46.077 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:25:47.491 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:25:47.595 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:25:48.450 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:25:48.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:25:48.513 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:25:48.540 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:25:48.586 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:25:48.622 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:25:48.656 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:25:48.682 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:25:48.689 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:25:48.690 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:25:48.690 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:25:48.690 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:25:48.690 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:25:48.709 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:25:48.735 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:25:48.736 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:25:48.737 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:25:48.737 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:25:48.737 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:25:48.737 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:25:48.737 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:25:48.737 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:25:48.737 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:25:48.738 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:25:48.738 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:25:48.738 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:25:48.738 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:25:48.738 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:25:49.524 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:25:49.818 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:25:49.818 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:25:50.714 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:25:51.762 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:25:51.763 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:25:51.763 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:25:51.763 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:25:51.763 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:25:51.820 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:25:52.412 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:25:53.357 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:25:54.361 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:25:54.393 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:25:54.422 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:25:54.448 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:25:54.476 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:25:54.509 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:25:54.540 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:25:54.567 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:25:54.595 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:25:54.623 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:25:54.649 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:25:54.651 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:25:54.651 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:25:54.651 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:25:54.651 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:25:54.651 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:25:54.651 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:25:54.652 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:25:54.652 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:25:54.652 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:25:54.652 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:25:54.652 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:25:54.652 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:25:54.653 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:25:54.883 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:25:55.357 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:25:55.358 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:25:55.798 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:25:56.830 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:25:57.359 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:25:58.093 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:25:58.652 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:25:59.666 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:25:59.932 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:25:59.966 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:25:59.992 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:26:00.026 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:26:00.051 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:26:00.077 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:26:00.103 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:26:00.129 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:26:00.154 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:26:00.182 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:26:00.209 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:26:00.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:26:00.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:26:00.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:26:00.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:26:00.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:26:00.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:26:00.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:26:00.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:26:00.212 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:26:00.213 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:26:00.213 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:26:00.213 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:26:00.213 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:26:00.603 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:26:00.603 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:26:00.603 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:26:00.603 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:26:00.603 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:26:00.982 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:26:00.982 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:26:01.411 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:26:02.797 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:26:02.984 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:26:03.802 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:26:03.802 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:26:03.803 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:26:03.803 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:26:04.344 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:26:05.268 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:26:05.526 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:26:05.556 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:26:05.574 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:26:05.597 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:26:05.623 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:26:05.649 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:26:05.675 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:26:05.704 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:26:05.731 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:26:05.757 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:26:05.783 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:26:05.785 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:26:05.785 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:26:05.785 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:26:05.785 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:26:05.785 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:26:05.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:26:05.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:26:05.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:26:05.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:26:05.786 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:26:05.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:26:05.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:26:05.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:26:06.560 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:26:06.560 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:26:07.167 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:26:07.791 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:26:08.317 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:26:08.561 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:26:10.372 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:26:10.942 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:26:10.977 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:26:11.295 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:26:11.317 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:26:11.341 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:26:11.366 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:26:11.392 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:26:11.419 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:26:11.446 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:26:11.472 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:26:11.499 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:26:11.526 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:26:11.552 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:26:11.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:26:11.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:26:11.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:26:11.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:26:11.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:26:11.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:26:11.556 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:26:11.556 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:26:11.556 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:26:11.556 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:26:11.556 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:26:11.556 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:26:11.556 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:26:12.202 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:26:12.202 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:26:13.638 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:26:14.123 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:26:14.155 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:26:14.155 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:26:14.203 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:26:16.725 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:26:16.773 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:26:16.848 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:26:17.587 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:26:17.588 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:26:17.627 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:26:17.673 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:26:17.706 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:26:17.745 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:26:17.777 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:26:17.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:26:17.834 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:26:17.863 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:26:17.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:26:17.921 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:26:17.922 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:26:17.922 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:26:17.922 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:26:17.922 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:26:17.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:26:17.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:26:17.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:26:17.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:26:17.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:26:17.923 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:26:17.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:26:17.924 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:26:17.924 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:26:18.988 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:26:18.988 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:26:20.145 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:26:20.647 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:26:20.648 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:26:20.820 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:26:20.989 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:26:23.253 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:26:23.353 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:26:23.517 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:26:24.113 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:26:24.113 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:26:24.206 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:26:24.240 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:26:24.269 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:26:24.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:26:24.325 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:26:24.368 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:26:24.401 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:26:24.420 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:26:24.446 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:26:24.471 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:26:24.473 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:26:24.473 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:26:24.473 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:26:24.473 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:26:24.474 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:26:24.474 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:26:24.474 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:26:24.474 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:26:24.474 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:26:24.474 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:26:24.474 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:26:24.474 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:26:24.475 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:26:25.481 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:26:25.482 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:26:26.464 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:26:26.876 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:26:27.031 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:26:27.031 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:26:27.482 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:26:29.402 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:26:29.433 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:26:29.658 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:26:29.679 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:26:29.702 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:26:29.725 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:26:29.751 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:26:29.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:26:29.804 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:26:29.831 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:26:29.857 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:26:29.884 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:26:29.911 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:26:29.913 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:26:29.913 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:26:29.913 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:26:29.913 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:26:29.913 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:26:29.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:26:29.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:26:29.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:26:29.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:26:29.914 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:26:29.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:26:29.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:26:29.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:26:29.955 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:26:29.955 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:26:30.595 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:26:30.596 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:26:32.457 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:26:32.596 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:26:32.947 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:26:32.947 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:26:33.114 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:26:35.148 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:26:35.175 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:26:35.199 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:26:35.223 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:26:35.253 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:26:35.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:26:35.336 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:26:35.373 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:26:35.414 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:26:35.452 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:26:35.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:26:35.486 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:26:35.486 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:26:35.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:26:35.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:26:35.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:26:35.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:26:35.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:26:35.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:26:35.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:26:35.488 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:26:35.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:26:35.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:26:35.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:26:35.627 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:26:35.666 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:26:36.468 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:26:36.468 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:26:36.645 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:26:36.646 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:26:38.469 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:26:38.823 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:26:38.840 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:26:39.691 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:26:39.692 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:26:41.068 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:26:41.099 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:26:41.125 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:26:41.157 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:26:41.178 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:26:41.204 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:26:41.229 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:26:41.255 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:26:41.281 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:26:41.309 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:26:41.335 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:26:41.338 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:26:41.338 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:26:41.338 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:26:41.338 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:26:41.338 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:26:41.339 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:26:41.339 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:26:41.339 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:26:41.339 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:26:41.339 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:26:41.339 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:26:41.339 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:26:41.340 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:26:41.845 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:26:41.988 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:26:41.989 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:26:42.271 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:26:42.902 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:26:42.903 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:26:43.990 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:26:44.853 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:26:45.493 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:26:46.291 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:26:46.291 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:26:46.696 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:26:46.721 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:26:46.746 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:26:46.769 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:26:46.797 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:26:46.824 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:26:46.851 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:26:46.885 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:26:46.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:26:46.931 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:26:46.957 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:26:46.959 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:26:46.960 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:26:46.960 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:26:46.960 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:26:46.960 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:26:46.960 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:26:46.960 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:26:46.961 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:26:46.961 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:26:46.961 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:26:46.961 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:26:46.961 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:26:46.961 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:26:47.613 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:26:47.613 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:26:47.653 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:26:48.387 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:26:48.944 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:26:48.944 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:26:49.615 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:26:50.835 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:26:52.020 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:26:52.428 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:26:52.454 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:26:52.478 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:26:52.502 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:26:52.531 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:26:52.540 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:26:52.541 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:26:52.580 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:26:52.620 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:26:52.658 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:26:52.685 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:26:52.712 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:26:52.738 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:26:52.740 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:26:52.740 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:26:52.740 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:26:52.740 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:26:52.740 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:26:52.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:26:52.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:26:52.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:26:52.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:26:52.741 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:26:52.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:26:52.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:26:52.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:26:53.399 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:26:53.400 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:26:53.672 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:26:54.815 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:26:55.400 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:26:55.571 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:26:55.572 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:26:56.725 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:26:58.258 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:26:58.293 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:26:58.324 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:26:58.349 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:26:58.381 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:26:58.407 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:26:58.433 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:26:58.448 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:26:58.459 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:26:58.488 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:26:58.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:26:58.543 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:26:58.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:26:58.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:26:58.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:26:58.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:26:58.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:26:58.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:26:58.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:26:58.546 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:26:58.546 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:26:58.546 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:26:58.546 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:26:58.546 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:26:58.546 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:26:59.092 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:26:59.093 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:26:59.244 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:26:59.244 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:26:59.787 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:27:01.246 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:27:01.428 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:27:02.123 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:27:02.123 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:27:02.832 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:27:04.444 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:27:04.469 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:27:04.506 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:27:04.541 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:27:04.577 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:27:04.604 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:27:04.630 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:27:04.657 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:27:04.670 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:27:04.687 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:27:04.714 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:27:04.757 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:27:04.758 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:27:04.758 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:27:04.758 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:27:04.759 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:27:04.759 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:27:04.759 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:27:04.759 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:27:04.759 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:27:04.759 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:27:04.759 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:27:04.760 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:27:04.760 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:27:04.760 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:27:05.411 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:27:05.411 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:27:05.588 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:27:05.588 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:27:05.739 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:27:07.590 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:27:07.773 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:27:08.498 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:27:08.499 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:27:09.165 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:27:10.856 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:27:10.888 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:27:10.916 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:27:10.942 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:27:10.968 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:27:10.995 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:27:11.021 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:27:11.047 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:27:11.090 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:27:11.113 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:27:11.130 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:27:11.160 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:27:11.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:27:11.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:27:11.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:27:11.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:27:11.162 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:27:11.162 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:27:11.162 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:27:11.162 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:27:11.162 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:27:11.162 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:27:11.163 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:27:11.163 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:27:11.163 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:27:11.748 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:27:11.748 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:27:11.896 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:27:11.897 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:27:11.934 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:27:13.897 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:27:14.038 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:27:14.588 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:27:14.588 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:27:15.184 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:27:16.993 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:27:17.015 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:27:17.039 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:27:17.062 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:27:17.088 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:27:17.114 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:27:17.140 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:27:17.143 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:27:17.167 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:27:17.194 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:27:17.221 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:27:17.247 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:27:17.247 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:27:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:27:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:27:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:27:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:27:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:27:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:27:17.248 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:27:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:27:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:27:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:27:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:27:17.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:27:17.827 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:27:17.828 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:27:17.983 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:27:17.983 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:27:18.265 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:27:19.983 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:27:20.266 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:27:20.856 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:27:20.957 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:27:22.795 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:27:22.827 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:27:22.880 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:27:22.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:27:22.935 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:27:22.961 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:27:22.987 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:27:23.012 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:27:23.039 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:27:23.066 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:27:23.093 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:27:23.095 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:27:23.095 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:27:23.095 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:27:23.095 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:27:23.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:27:23.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:27:23.097 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:27:23.097 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:27:23.097 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:27:23.097 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:27:23.098 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:27:23.098 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:27:23.098 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:27:23.352 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:27:23.353 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:27:23.353 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:27:23.353 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:27:23.353 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:27:23.393 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:27:23.925 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:27:23.925 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:27:24.104 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:27:25.926 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:27:26.361 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:27:26.361 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:27:26.361 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:27:26.361 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:27:26.361 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:27:26.903 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:27:27.014 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:27:28.545 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:27:28.571 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:27:28.597 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:27:28.621 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:27:28.648 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:27:28.675 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:27:28.701 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:27:28.727 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:27:28.771 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:27:28.810 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:27:28.849 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:27:28.851 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:27:28.851 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:27:28.851 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:27:28.851 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:27:28.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:27:28.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:27:28.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:27:28.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:27:28.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:27:28.852 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:27:28.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:27:28.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:27:28.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:27:29.504 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:27:29.504 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:27:29.631 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:27:29.779 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:27:30.465 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:27:31.505 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:27:32.915 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:27:33.117 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:27:33.775 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:27:34.451 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:27:34.475 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:27:34.497 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:27:34.529 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:27:34.556 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:27:34.582 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:27:34.607 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:27:34.634 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:27:34.661 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:27:34.688 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:27:34.713 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:27:34.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:27:34.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:27:34.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:27:34.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:27:34.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:27:34.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:27:34.716 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:27:34.716 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:27:34.716 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:27:34.716 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:27:34.716 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:27:34.716 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:27:34.716 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:27:35.477 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:27:35.477 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:27:36.004 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:27:36.004 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:27:36.004 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:27:36.005 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:27:36.005 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:27:36.044 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:27:36.673 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:27:37.478 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:27:39.534 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:27:39.534 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:27:39.535 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:27:39.535 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:27:39.873 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:27:40.343 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:27:40.438 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:27:40.480 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:27:40.509 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:27:40.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:27:40.561 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:27:40.588 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:27:40.615 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:27:40.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:27:40.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:27:40.709 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:27:40.746 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:27:40.748 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:27:40.748 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:27:40.748 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:27:40.748 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:27:40.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:27:40.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:27:40.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:27:40.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:27:40.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:27:40.749 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:27:40.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:27:40.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:27:40.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:27:41.351 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:27:41.351 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:27:42.719 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:27:42.752 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:27:43.181 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:27:43.353 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:27:45.876 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:27:46.419 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:27:46.460 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:27:46.487 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:27:46.513 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:27:46.516 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:27:46.516 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:27:46.516 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:27:46.517 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:27:46.517 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:27:46.541 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:27:46.568 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:27:46.595 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:27:46.622 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:27:46.648 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:27:46.678 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:27:46.702 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:27:46.703 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:27:46.704 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:27:46.704 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:27:46.704 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:27:46.704 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:27:46.704 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:27:46.704 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:27:46.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:27:46.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:27:46.705 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:27:46.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:27:46.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:27:46.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:27:47.021 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:27:47.325 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:27:47.326 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:27:48.453 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:27:48.948 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:27:48.948 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:27:48.949 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:27:48.949 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:27:49.327 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:27:49.500 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:27:51.001 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:27:52.490 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:27:52.516 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:27:52.540 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:27:52.565 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:27:52.592 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:27:52.625 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:27:52.633 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:27:52.651 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:27:52.678 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:27:52.705 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:27:52.732 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:27:52.759 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:27:52.760 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:27:52.760 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:27:52.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:27:52.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:27:52.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:27:52.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:27:52.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:27:52.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:27:52.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:27:52.762 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:27:52.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:27:52.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:27:52.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:27:53.120 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:27:53.504 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:27:53.505 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:27:54.005 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:27:55.471 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:27:55.505 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:27:56.118 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:27:56.727 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:27:58.357 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:27:58.382 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:27:58.412 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:27:58.435 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:27:58.462 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:27:58.488 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:27:58.518 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:27:58.549 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:27:58.553 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:27:58.576 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:27:58.603 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:27:58.630 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:27:58.632 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:27:58.632 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:27:58.632 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:27:58.632 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:27:58.633 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:27:58.633 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:27:58.633 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:27:58.633 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:27:58.633 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:27:58.633 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:27:58.633 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:27:58.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:27:58.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:27:59.203 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:27:59.204 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:27:59.367 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:27:59.368 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:27:59.665 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:28:01.369 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:28:01.429 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:28:02.341 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:28:02.341 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:28:02.868 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:28:04.528 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:28:04.565 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:28:04.594 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:28:04.618 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:28:04.645 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:28:04.680 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:28:04.715 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:28:04.751 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:28:04.786 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:28:04.815 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:28:04.826 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:28:04.843 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:28:04.844 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:28:04.844 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:28:04.844 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:28:04.844 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:28:04.844 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:28:04.845 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:28:04.845 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:28:04.845 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:28:04.845 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:28:04.845 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:28:04.845 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:28:04.845 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:28:04.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:28:05.481 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:28:05.481 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:28:05.635 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:28:05.636 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:28:05.753 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:28:07.636 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:28:07.743 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:28:08.387 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:28:08.388 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:28:09.145 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:28:10.746 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:28:10.767 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:28:10.791 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:28:10.815 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:28:10.842 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:28:10.869 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:28:10.895 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:28:10.922 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:28:10.947 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:28:10.960 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:28:10.974 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:28:11.000 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:28:11.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:28:11.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:28:11.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:28:11.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:28:11.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:28:11.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:28:11.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:28:11.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:28:11.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:28:11.003 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:28:11.003 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:28:11.003 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:28:11.003 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:28:11.616 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:28:11.616 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:28:11.768 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:28:11.769 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:28:11.832 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:28:13.770 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:28:13.846 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:28:14.465 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:28:14.465 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:28:15.170 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:28:16.793 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:28:16.820 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:28:16.845 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:28:16.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:28:16.894 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:28:16.921 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:28:16.948 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:28:16.973 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:28:16.999 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:28:17.027 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:28:17.049 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:28:17.053 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:28:17.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:28:17.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:28:17.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:28:17.056 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:28:17.056 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:28:17.056 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:28:17.056 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:28:17.056 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:28:17.056 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:28:17.057 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:28:17.057 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:28:17.057 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:28:17.057 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:28:17.702 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:28:17.702 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:28:17.852 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:28:17.852 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:28:18.091 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:28:19.842 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:28:19.854 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:28:20.472 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:28:20.472 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:28:21.054 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:28:22.808 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:28:22.833 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:28:22.858 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:28:22.888 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:28:22.917 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:28:22.943 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:28:22.971 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:28:22.998 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:28:23.025 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:28:23.042 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:28:23.057 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:28:23.085 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:28:23.086 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:28:23.086 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:28:23.086 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:28:23.086 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:28:23.087 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:28:23.087 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:28:23.087 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:28:23.087 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:28:23.087 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:28:23.087 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:28:23.087 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:28:23.088 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:28:23.088 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:28:23.680 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:28:23.681 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:28:23.829 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:28:23.830 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:28:24.121 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:28:25.830 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:28:25.908 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:28:26.448 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:28:26.449 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:28:27.100 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:28:29.024 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:28:29.051 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:28:29.051 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:28:29.079 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:28:29.101 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:28:29.129 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:28:29.158 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:28:29.184 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:28:29.212 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:28:29.236 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:28:29.262 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:28:29.289 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:28:29.290 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:28:29.291 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:28:29.291 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:28:29.291 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:28:29.292 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:28:29.292 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:28:29.292 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:28:29.292 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:28:29.293 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:28:29.293 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:28:29.293 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:28:29.293 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:28:29.572 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:28:29.573 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:28:29.978 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:28:29.978 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:28:30.142 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:28:31.845 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:28:31.979 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:28:32.678 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:28:32.679 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:28:33.889 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:28:35.011 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:28:35.038 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:28:35.063 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:28:35.088 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:28:35.114 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:28:35.147 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:28:35.177 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:28:35.207 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:28:35.210 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:28:35.234 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:28:35.261 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:28:35.287 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:28:35.289 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:28:35.289 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:28:35.290 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:28:35.290 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:28:35.290 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:28:35.290 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:28:35.290 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:28:35.290 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:28:35.291 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:28:35.291 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:28:35.291 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:28:35.291 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:28:35.291 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:28:35.913 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:28:35.914 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:28:36.078 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:28:36.078 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:28:36.798 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:28:38.079 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:28:38.135 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:28:38.773 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:28:38.773 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:28:39.908 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:28:41.243 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:28:41.266 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:28:41.303 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:28:41.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:28:41.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:28:41.391 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:28:41.417 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:28:41.443 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:28:41.459 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:28:41.481 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:28:41.504 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:28:41.530 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:28:41.532 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:28:41.533 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:28:41.533 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:28:41.533 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:28:41.533 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:28:41.533 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:28:41.533 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:28:41.533 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:28:41.534 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:28:41.534 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:28:41.534 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:28:41.534 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:28:41.534 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:28:42.122 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:28:42.123 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:28:42.269 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:28:42.269 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:28:42.772 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:28:44.270 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:28:44.381 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:28:45.020 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:28:45.021 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:28:45.909 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:28:47.369 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:28:47.645 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:28:48.227 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:28:48.227 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:28:48.231 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:28:48.276 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:28:48.317 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:28:48.350 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:28:48.379 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:28:48.404 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:28:48.431 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:28:48.457 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:28:48.484 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:28:48.511 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:28:48.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:28:48.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:28:48.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:28:48.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:28:48.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:28:48.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:28:48.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:28:48.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:28:48.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:28:48.514 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:28:48.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:28:48.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:28:48.515 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:28:48.743 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:28:49.110 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:28:49.111 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:28:50.850 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:28:51.111 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:28:51.403 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:28:51.403 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:28:52.118 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:28:53.882 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:28:54.089 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:28:54.659 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:28:54.660 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:28:54.742 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:28:54.784 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:28:54.820 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:28:54.855 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:28:54.888 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:28:54.919 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:28:54.949 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:28:54.979 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:28:55.007 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:28:55.033 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:28:55.035 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:28:55.035 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:28:55.035 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:28:55.036 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:28:55.036 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:28:55.036 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:28:55.036 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:28:55.036 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:28:55.037 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:28:55.037 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:28:55.037 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:28:55.037 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:28:55.037 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:28:55.106 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:28:55.856 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:28:55.856 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:28:57.121 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:28:57.850 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:28:57.851 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:28:57.857 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:28:58.911 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:29:00.348 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:29:00.879 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:29:00.879 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:29:00.879 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:29:00.901 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:29:00.926 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:29:00.953 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:29:00.981 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:29:01.004 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:29:01.029 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:29:01.055 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:29:01.081 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:29:01.126 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:29:01.152 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:29:01.153 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:29:01.154 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:29:01.154 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:29:01.154 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:29:01.154 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:29:01.154 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:29:01.154 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:29:01.155 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:29:01.155 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:29:01.155 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:29:01.155 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:29:01.155 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:29:01.155 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:29:01.692 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:29:01.965 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:29:01.965 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:29:03.105 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:29:03.106 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:29:03.106 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:29:03.106 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:29:03.851 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:29:03.967 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:29:04.917 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:29:06.494 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:29:06.494 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:29:06.494 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:29:06.494 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:29:06.494 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:29:06.974 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:29:07.009 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:29:07.043 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:29:07.043 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:29:07.070 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:29:07.099 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:29:07.124 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:29:07.151 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:29:07.177 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:29:07.203 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:29:07.230 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:29:07.256 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:29:07.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:29:07.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:29:07.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:29:07.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:29:07.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:29:07.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:29:07.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:29:07.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:29:07.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:29:07.259 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:29:07.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:29:07.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:29:07.260 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:29:07.826 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:29:07.899 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:29:07.900 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:29:09.630 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:29:09.900 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:29:10.217 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:29:11.021 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:29:12.644 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:29:12.680 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:29:12.711 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:29:12.739 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:29:12.768 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:29:12.795 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:29:12.822 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:29:12.848 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:29:12.878 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:29:12.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:29:12.925 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:29:12.925 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:29:12.925 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:29:12.925 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:29:12.941 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:29:12.942 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:29:12.943 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:29:12.943 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:29:12.943 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:29:12.943 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:29:12.943 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:29:12.943 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:29:12.944 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:29:12.944 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:29:12.944 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:29:12.944 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:29:12.944 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:29:12.944 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:29:13.601 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:29:13.765 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:29:13.766 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:29:13.923 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:29:15.767 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:29:15.894 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:29:15.894 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:29:15.894 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:29:15.895 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:29:15.895 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:29:16.414 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:29:16.784 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:29:18.849 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:29:19.145 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:29:19.656 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:29:19.710 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:29:19.763 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:29:19.791 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:29:19.836 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:29:19.869 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:29:19.897 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:29:19.924 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:29:19.955 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:29:19.979 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:29:20.007 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:29:20.008 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:29:20.008 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:29:20.008 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:29:20.009 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:29:20.009 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:29:20.009 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:29:20.009 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:29:20.009 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:29:20.009 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:29:20.010 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:29:20.010 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:29:20.010 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:29:20.010 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:29:20.115 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:29:20.558 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:29:20.558 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:29:22.280 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:29:22.559 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:29:22.783 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:29:23.924 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:29:25.249 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:29:25.540 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:29:26.077 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:29:26.105 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:29:26.151 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:29:26.188 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:29:26.221 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:29:26.254 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:29:26.280 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:29:26.308 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:29:26.334 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:29:26.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:29:26.388 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:29:26.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:29:26.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:29:26.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:29:26.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:29:26.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:29:26.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:29:26.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:29:26.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:29:26.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:29:26.391 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:29:26.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:29:26.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:29:26.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:29:26.524 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:29:26.999 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:29:26.999 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:29:28.556 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:29:28.557 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:29:28.557 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:29:28.557 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:29:28.557 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:29:29.000 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:29:29.109 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:29:29.303 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:29:31.527 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:29:31.550 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:29:31.575 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:29:31.598 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:29:31.624 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:29:31.650 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:29:31.677 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:29:31.704 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:29:31.732 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:29:31.757 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:29:31.784 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:29:31.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:29:31.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:29:31.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:29:31.786 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:29:31.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:29:31.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:29:31.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:29:31.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:29:31.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:29:31.788 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:29:31.788 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:29:31.788 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:29:31.788 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:29:32.209 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:29:32.249 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:29:32.339 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:29:32.340 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:29:32.693 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:29:34.341 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:29:35.237 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:29:35.348 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:29:36.076 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:29:36.842 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:29:36.866 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:29:36.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:29:36.913 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:29:36.942 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:29:36.968 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:29:36.995 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:29:37.021 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:29:37.048 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:29:37.078 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:29:37.115 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:29:37.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:29:37.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:29:37.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:29:37.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:29:37.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:29:37.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:29:37.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:29:37.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:29:37.117 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:29:37.117 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:29:37.117 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:29:37.117 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:29:37.117 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:29:37.910 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:29:37.910 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:29:38.235 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:29:38.373 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:29:38.373 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:29:38.373 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:29:38.373 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:29:38.373 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:29:38.898 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:29:39.911 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:29:41.826 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:29:42.118 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:29:42.381 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:29:42.382 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:29:42.818 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:29:42.865 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:29:42.894 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:29:42.923 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:29:42.951 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:29:42.978 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:29:43.003 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:29:43.028 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:29:43.053 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:29:43.082 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:29:43.108 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:29:43.111 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:29:43.111 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:29:43.111 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:29:43.111 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:29:43.111 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:29:43.111 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:29:43.112 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:29:43.112 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:29:43.112 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:29:43.112 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:29:43.112 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:29:43.112 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:29:43.112 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:29:43.653 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:29:43.653 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:29:44.684 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:29:44.873 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:29:45.194 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:29:45.194 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:29:45.654 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:29:48.055 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:29:48.612 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:29:48.612 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:29:48.694 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:29:48.932 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:29:49.546 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:29:49.588 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:29:49.624 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:29:49.653 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:29:49.680 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:29:49.741 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:29:49.782 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:29:49.811 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:29:49.840 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:29:49.870 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:29:49.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:29:49.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:29:49.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:29:49.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:29:49.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:29:49.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:29:49.873 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:29:49.873 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:29:49.873 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:29:49.873 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:29:49.873 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:29:49.873 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:29:49.874 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:29:50.432 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:29:50.432 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:29:50.943 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:29:51.457 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:29:51.457 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:29:51.721 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:29:52.434 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:29:53.966 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:29:54.811 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:29:54.812 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:29:55.169 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:29:55.343 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:29:56.197 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:29:56.245 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:29:56.279 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:29:56.312 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:29:56.340 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:29:56.369 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:29:56.396 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:29:56.423 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:29:56.450 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:29:56.477 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:29:56.480 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:29:56.480 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:29:56.480 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:29:56.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:29:56.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:29:56.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:29:56.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:29:56.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:29:56.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:29:56.481 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:29:56.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:29:56.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:29:56.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:29:56.985 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:29:57.027 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:29:57.028 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:29:57.443 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:29:57.444 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:29:57.847 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:29:59.028 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:29:59.845 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:30:00.532 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:30:00.532 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:30:00.952 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:30:02.015 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:30:02.877 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:30:02.930 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:30:02.997 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:30:03.025 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:30:03.051 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:30:03.054 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:30:03.081 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:30:03.112 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:30:03.148 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:30:03.181 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:30:03.213 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:30:03.214 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:30:03.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:30:03.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:30:03.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:30:03.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:30:03.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:30:03.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:30:03.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:30:03.216 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:30:03.216 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:30:03.216 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:30:03.216 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:30:03.216 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:30:03.542 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:30:03.542 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:30:03.888 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:30:03.889 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:30:03.952 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:30:05.890 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:30:05.913 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:30:06.539 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:30:06.539 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:30:06.968 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:30:08.850 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:30:08.889 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:30:08.921 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:30:08.945 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:30:08.972 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:30:08.998 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:30:09.024 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:30:09.050 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:30:09.077 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:30:09.112 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:30:09.112 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:30:09.130 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:30:09.131 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:30:09.132 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:30:09.132 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:30:09.132 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:30:09.132 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:30:09.132 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:30:09.132 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:30:09.132 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:30:09.133 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:30:09.133 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:30:09.133 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:30:09.133 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:30:09.133 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:30:09.756 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:30:09.757 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:30:09.913 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:30:09.913 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:30:10.254 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:30:11.914 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:30:12.108 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:30:12.971 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:30:12.972 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:30:14.020 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:30:14.687 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:30:14.705 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:30:14.728 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:30:14.752 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:30:14.779 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:30:14.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:30:14.838 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:30:14.865 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:30:14.891 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:30:14.918 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:30:14.947 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:30:14.947 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:30:14.948 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:30:14.948 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:30:14.948 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:30:14.948 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:30:14.948 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:30:14.948 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:30:14.949 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:30:14.949 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:30:14.949 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:30:14.949 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:30:14.949 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:30:14.949 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:30:15.581 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:30:15.581 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:30:15.962 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:30:16.520 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:30:16.520 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:30:16.821 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:30:17.582 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:30:19.356 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:30:20.189 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:30:20.190 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:30:20.512 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:30:20.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:30:20.558 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:30:20.588 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:30:20.617 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:30:20.645 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:30:20.673 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:30:20.703 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:30:20.730 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:30:20.758 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:30:20.785 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:30:20.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:30:20.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:30:20.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:30:20.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:30:20.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:30:20.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:30:20.787 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:30:20.788 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:30:20.788 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:30:20.788 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:30:20.788 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:30:20.788 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:30:20.788 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:30:20.958 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:30:21.362 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:30:21.362 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:30:22.588 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:30:23.117 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:30:23.118 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:30:23.363 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:30:23.831 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:30:26.031 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:30:26.066 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:30:26.096 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:30:26.096 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:30:26.123 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:30:26.152 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:30:26.180 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:30:26.206 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:30:26.235 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:30:26.262 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:30:26.288 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:30:26.317 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:30:26.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:30:26.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:30:26.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:30:26.320 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:30:26.320 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:30:26.320 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:30:26.320 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:30:26.320 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:30:26.320 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:30:26.320 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:30:26.321 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:30:26.321 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:30:26.321 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:30:26.695 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:30:26.695 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:30:27.293 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:30:27.293 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:30:27.953 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:30:29.080 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:30:29.294 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:30:29.664 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:30:29.664 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:30:30.649 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:30:31.582 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:30:31.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:30:31.628 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:30:31.655 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:30:31.680 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:30:31.707 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:30:31.735 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:30:31.761 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:30:31.793 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:30:31.824 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:30:31.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:30:31.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:30:31.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:30:31.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:30:31.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:30:31.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:30:31.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:30:31.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:30:31.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:30:31.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:30:31.855 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:30:31.855 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:30:31.855 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:30:31.855 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:30:31.900 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:30:32.571 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:30:32.571 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:30:32.915 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:30:32.915 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:30:34.125 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:30:34.699 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:30:34.916 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:30:35.162 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:30:35.163 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:30:36.782 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:30:37.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:30:37.311 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:30:37.316 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:30:37.334 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:30:37.357 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:30:37.383 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:30:37.409 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:30:37.435 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:30:37.461 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:30:37.487 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:30:37.514 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:30:37.542 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:30:37.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:30:37.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:30:37.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:30:37.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:30:37.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:30:37.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:30:37.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:30:37.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:30:37.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:30:37.545 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:30:37.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:30:37.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:30:37.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:30:37.966 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:30:37.966 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:30:38.398 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:30:38.398 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:30:39.963 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:30:40.399 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:30:40.753 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:30:41.536 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:30:41.537 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:30:42.545 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:30:42.698 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:30:43.550 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:30:43.595 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:30:43.623 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:30:43.655 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:30:43.681 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:30:43.713 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:30:43.733 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:30:43.760 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:30:43.787 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:30:43.789 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:30:43.813 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:30:43.814 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:30:43.815 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:30:43.815 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:30:43.815 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:30:43.815 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:30:43.816 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:30:43.816 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:30:43.816 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:30:43.816 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:30:43.817 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:30:43.817 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:30:43.817 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:30:43.817 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:30:44.704 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:30:44.704 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:30:44.712 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:30:44.713 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:30:45.542 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:30:46.715 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:30:46.869 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:30:47.543 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:30:48.343 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:30:49.093 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:30:49.123 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:30:49.153 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:30:49.178 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:30:49.205 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:30:49.232 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:30:49.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:30:49.331 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:30:49.373 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:30:49.408 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:30:49.440 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:30:49.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:30:49.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:30:49.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:30:49.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:30:49.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:30:49.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:30:49.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:30:49.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:30:49.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:30:49.442 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:30:49.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:30:49.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:30:49.444 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:30:50.346 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:30:50.346 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:30:50.346 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:30:50.346 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:30:50.347 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:30:50.556 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:30:50.557 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:30:51.033 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:30:52.557 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:30:52.594 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:30:53.312 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:30:53.312 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:30:53.312 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:30:53.313 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:30:53.313 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:30:53.865 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:30:55.174 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:30:55.174 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:30:55.174 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:30:55.174 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:30:55.175 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:30:55.175 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:30:55.175 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:30:55.197 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:30:55.224 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:30:55.250 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:30:55.276 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:30:55.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:30:55.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:30:55.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:30:55.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:30:55.279 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:30:55.279 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:30:55.279 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:30:55.279 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:30:55.279 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:30:55.279 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:30:55.279 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:30:55.280 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:30:55.280 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:30:55.294 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:30:55.980 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:30:55.981 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:30:56.190 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:30:56.779 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:30:57.982 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:30:58.984 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:30:59.325 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:31:00.130 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:31:00.668 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:31:00.691 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:31:00.716 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:31:00.741 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:31:00.767 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:31:00.794 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:31:00.822 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:31:00.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:31:00.879 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:31:00.907 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:31:00.933 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:31:00.935 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:31:00.935 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:31:00.935 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:31:00.935 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:31:00.935 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:31:00.936 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:31:00.936 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:31:00.936 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:31:00.936 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:31:00.936 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:31:00.936 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:31:00.936 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:31:00.937 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:31:01.687 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:31:01.687 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:31:02.129 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:31:02.460 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:31:02.461 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:31:02.461 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:31:02.461 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:31:02.461 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:31:03.273 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:31:03.687 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:31:04.997 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:31:06.149 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:31:06.149 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:31:06.150 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:31:06.150 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:31:06.826 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:31:06.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:31:06.865 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:31:06.877 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:31:06.913 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:31:06.939 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:31:06.969 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:31:06.996 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:31:07.021 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:31:07.047 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:31:07.075 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:31:07.102 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:31:07.103 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:31:07.103 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:31:07.104 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:31:07.104 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:31:07.104 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:31:07.104 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:31:07.104 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:31:07.104 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:31:07.104 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:31:07.105 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:31:07.105 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:31:07.105 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:31:07.105 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:31:07.628 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:31:07.817 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:31:07.818 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:31:08.975 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:31:09.733 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:31:09.819 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:31:11.027 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:31:12.358 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:31:12.877 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:31:12.900 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:31:12.929 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:31:12.940 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:31:12.953 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:31:12.980 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:31:13.006 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:31:13.032 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:31:13.063 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:31:13.090 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:31:13.118 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:31:13.146 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:31:13.147 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:31:13.147 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:31:13.147 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:31:13.147 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:31:13.147 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:31:13.148 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:31:13.148 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:31:13.148 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:31:13.148 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:31:13.148 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:31:13.148 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:31:13.148 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:31:13.149 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:31:13.666 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:31:13.757 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:31:13.758 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:31:15.481 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:31:15.482 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:31:15.482 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:31:15.482 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:31:15.759 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:31:16.085 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:31:16.769 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:31:18.282 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:31:18.311 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:31:18.336 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:31:18.360 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:31:18.386 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:31:18.413 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:31:18.438 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:31:18.465 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:31:18.491 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:31:18.518 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:31:18.544 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:31:18.546 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:31:18.546 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:31:18.546 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:31:18.546 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:31:18.547 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:31:18.547 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:31:18.547 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:31:18.547 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:31:18.547 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:31:18.547 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:31:18.547 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:31:18.548 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:31:18.548 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:31:18.583 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:31:19.198 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:31:19.198 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:31:19.353 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:31:19.589 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:31:21.199 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:31:21.619 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:31:22.114 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:31:22.405 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:31:24.076 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:31:24.113 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:31:24.145 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:31:24.171 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:31:24.198 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:31:24.224 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:31:24.251 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:31:24.278 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:31:24.304 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:31:24.342 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:31:24.377 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:31:24.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:31:24.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:31:24.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:31:24.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:31:24.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:31:24.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:31:24.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:31:24.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:31:24.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:31:24.380 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:31:24.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:31:24.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:31:24.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:31:24.906 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:31:24.974 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:31:24.974 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:31:25.021 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:31:25.591 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:31:25.591 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:31:26.975 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:31:28.299 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:31:28.477 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:31:29.215 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:31:29.216 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:31:29.625 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:31:29.648 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:31:29.671 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:31:29.694 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:31:29.721 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:31:29.747 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:31:29.797 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:31:29.837 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:31:29.874 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:31:29.904 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:31:29.931 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:31:29.933 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:31:29.933 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:31:29.933 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:31:29.933 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:31:29.933 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:31:29.933 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:31:29.933 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:31:29.934 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:31:29.934 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:31:29.934 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:31:29.934 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:31:29.934 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:31:29.934 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:31:30.584 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:31:30.584 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:31:30.952 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:31:31.348 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:31:31.963 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:31:31.964 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:31:32.585 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:31:34.319 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:31:34.837 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:31:35.482 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:31:35.510 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:31:35.520 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:31:35.520 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:31:35.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:31:35.558 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:31:35.585 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:31:35.611 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:31:35.639 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:31:35.664 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:31:35.690 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:31:35.717 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:31:35.744 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:31:35.745 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:31:35.745 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:31:35.746 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:31:35.746 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:31:35.746 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:31:35.746 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:31:35.746 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:31:35.746 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:31:35.746 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:31:35.747 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:31:35.747 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:31:35.747 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:31:35.747 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:31:36.371 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:31:36.371 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:31:37.105 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:31:37.637 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:31:38.205 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:31:38.206 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:31:38.373 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:31:40.846 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:31:40.998 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:31:41.360 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:31:41.428 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:31:41.429 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:31:42.215 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:31:42.253 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:31:42.293 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:31:42.332 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:31:42.367 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:31:42.396 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:31:42.426 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:31:42.452 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:31:42.479 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:31:42.505 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:31:42.506 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:31:42.507 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:31:42.507 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:31:42.507 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:31:42.507 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:31:42.507 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:31:42.507 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:31:42.507 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:31:42.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:31:42.508 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:31:42.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:31:42.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:31:42.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:31:43.210 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:31:43.211 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:31:43.676 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:31:43.783 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:31:44.228 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:31:44.228 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:31:45.211 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:31:46.781 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:31:46.942 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:31:47.729 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:31:47.729 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:31:48.146 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:31:48.173 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:31:48.198 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:31:48.223 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:31:48.250 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:31:48.280 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:31:48.304 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:31:48.330 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:31:48.372 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:31:48.396 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:31:48.423 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:31:48.424 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:31:48.425 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:31:48.425 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:31:48.425 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:31:48.425 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:31:48.425 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:31:48.425 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:31:48.425 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:31:48.426 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:31:48.426 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:31:48.426 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:31:48.426 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:31:48.426 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:31:49.121 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:31:49.121 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:31:49.413 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:31:49.852 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:31:50.570 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:31:50.571 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:31:51.122 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:31:52.128 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:31:53.559 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:31:54.152 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:31:54.165 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:31:54.165 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:31:54.188 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:31:54.219 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:31:54.243 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:31:54.269 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:31:54.317 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:31:54.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:31:54.404 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:31:54.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:31:54.489 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:31:54.528 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:31:54.530 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:31:54.530 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:31:54.530 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:31:54.531 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:31:54.531 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:31:54.531 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:31:54.531 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:31:54.531 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:31:54.531 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:31:54.531 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:31:54.532 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:31:54.532 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:31:54.532 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:31:55.134 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:31:55.266 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:31:55.266 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:31:56.390 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:31:57.066 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:31:57.067 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:31:57.267 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:31:57.810 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:31:59.707 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:32:00.050 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:32:00.231 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:32:00.231 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:32:00.904 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:32:00.947 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:32:00.984 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:32:01.029 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:32:01.034 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:32:01.064 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:32:01.095 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:32:01.125 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:32:01.153 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:32:01.187 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:32:01.215 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:32:01.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:32:01.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:32:01.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:32:01.216 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:32:01.216 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:32:01.216 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:32:01.216 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:32:01.216 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:32:01.216 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:32:01.217 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:32:01.217 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:32:01.217 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:32:01.217 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:32:02.004 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:32:02.004 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:32:02.722 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:32:03.405 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:32:03.405 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:32:04.005 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:32:04.017 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:32:06.027 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:32:06.649 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:32:06.701 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:32:06.701 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:32:07.012 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:32:07.035 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:32:07.057 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:32:07.083 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:32:07.134 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:32:07.175 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:32:07.212 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:32:07.241 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:32:07.270 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:32:07.298 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:32:07.339 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:32:07.341 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:32:07.341 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:32:07.341 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:32:07.341 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:32:07.341 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:32:07.342 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:32:07.342 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:32:07.342 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:32:07.342 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:32:07.342 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:32:07.342 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:32:07.342 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:32:07.343 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:32:07.918 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:32:07.918 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:32:09.062 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:32:09.554 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:32:09.554 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:32:09.919 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:32:10.029 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:32:12.009 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:32:12.556 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:32:12.556 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:32:12.596 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:32:12.824 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:32:12.849 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:32:12.874 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:32:12.898 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:32:12.924 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:32:12.949 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:32:12.976 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:32:13.004 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:32:13.030 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:32:13.058 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:32:13.084 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:32:13.085 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:32:13.085 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:32:13.085 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:32:13.086 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:32:13.086 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:32:13.086 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:32:13.086 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:32:13.086 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:32:13.086 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:32:13.086 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:32:13.087 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:32:13.087 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:32:13.087 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:32:13.751 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:32:13.751 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:32:15.064 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:32:15.520 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:32:15.653 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:32:15.654 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:32:15.751 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:32:17.989 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:32:18.202 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:32:18.224 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:32:18.268 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:32:18.312 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:32:18.357 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:32:18.370 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:32:18.391 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:32:18.418 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:32:18.444 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:32:18.471 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:32:18.498 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:32:18.524 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:32:18.526 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:32:18.526 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:32:18.526 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:32:18.526 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:32:18.526 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:32:18.526 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:32:18.527 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:32:18.527 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:32:18.527 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:32:18.527 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:32:18.527 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:32:18.527 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:32:18.527 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:32:19.014 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:32:19.014 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:32:19.472 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:32:19.473 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:32:21.114 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:32:21.474 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:32:21.610 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:32:22.069 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:32:22.070 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:32:23.692 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:32:24.103 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:32:24.447 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:32:24.960 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:32:25.024 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:32:25.043 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:32:25.044 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:32:25.068 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:32:25.107 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:32:25.139 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:32:25.163 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:32:25.189 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:32:25.216 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:32:25.242 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:32:25.285 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:32:25.286 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:32:25.286 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:32:25.286 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:32:25.286 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:32:25.286 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:32:25.287 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:32:25.287 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:32:25.287 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:32:25.287 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:32:25.287 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:32:25.287 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:32:25.287 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:32:25.288 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:32:25.868 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:32:25.868 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:32:26.789 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:32:27.788 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:32:27.869 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:32:28.374 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:32:28.375 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:32:29.402 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:32:30.339 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:32:30.363 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:32:30.386 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:32:30.410 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:32:30.437 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:32:30.463 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:32:30.491 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:32:30.517 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:32:30.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:32:30.603 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:32:30.635 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:32:30.636 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:32:30.636 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:32:30.636 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:32:30.636 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:32:30.636 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:32:30.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:32:30.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:32:30.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:32:30.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:32:30.637 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:32:30.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:32:30.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:32:30.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:32:30.893 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:32:31.426 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:32:31.426 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:32:31.879 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:32:33.026 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:32:33.428 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:32:34.020 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:32:34.550 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:32:35.775 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:32:35.837 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:32:35.838 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:32:35.861 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:32:35.885 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:32:35.916 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:32:35.941 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:32:35.967 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:32:36.007 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:32:36.048 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:32:36.078 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:32:36.104 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:32:36.106 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:32:36.106 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:32:36.106 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:32:36.106 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:32:36.106 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:32:36.106 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:32:36.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:32:36.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:32:36.107 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:32:36.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:32:36.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:32:36.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:32:36.937 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:32:36.937 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:32:37.615 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:32:37.615 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:32:37.615 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:32:37.615 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:32:37.615 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:32:38.196 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:32:38.938 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:32:39.035 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:32:40.732 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:32:40.732 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:32:40.732 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:32:40.732 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:32:41.140 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:32:41.755 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:32:41.788 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:32:41.821 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:32:41.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:32:41.880 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:32:41.898 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:32:41.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:32:41.936 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:32:41.962 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:32:42.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:32:42.029 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:32:42.057 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:32:42.058 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:32:42.059 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:32:42.059 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:32:42.059 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:32:42.059 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:32:42.059 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:32:42.059 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:32:42.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:32:42.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:32:42.060 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:32:42.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:32:42.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:32:42.944 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:32:42.944 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:32:44.063 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:32:44.858 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:32:44.945 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:32:45.147 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:32:47.506 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:32:47.861 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:32:48.025 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:32:48.058 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:32:48.062 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:32:48.086 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:32:48.111 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:32:48.138 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:32:48.166 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:32:48.199 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:32:48.225 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:32:48.252 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:32:48.278 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:32:48.304 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:32:48.307 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:32:48.307 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:32:48.308 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:32:48.308 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:32:48.308 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:32:48.308 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:32:48.308 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:32:48.308 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:32:48.309 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:32:48.309 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:32:48.309 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:32:48.309 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:32:48.309 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:32:49.175 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:32:49.175 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:32:50.746 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:32:50.747 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:32:50.747 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:32:50.747 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:32:51.176 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:32:51.198 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:32:51.284 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:32:53.848 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:32:53.848 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:32:53.848 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:32:53.848 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:32:53.848 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:32:53.868 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:32:54.118 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:32:54.140 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:32:54.164 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:32:54.187 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:32:54.213 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:32:54.239 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:32:54.265 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:32:54.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:32:54.322 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:32:54.349 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:32:54.373 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:32:54.375 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:32:54.376 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:32:54.376 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:32:54.376 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:32:54.376 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:32:54.376 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:32:54.376 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:32:54.376 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:32:54.377 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:32:54.377 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:32:54.377 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:32:54.377 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:32:54.665 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:32:55.063 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:32:55.063 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:32:56.990 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:32:57.024 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:32:57.065 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:32:57.970 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:32:59.646 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:32:59.873 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:33:00.361 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:33:00.745 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:33:00.776 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:33:00.818 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:33:00.851 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:33:00.880 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:33:00.913 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:33:00.924 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:33:00.940 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:33:00.968 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:33:00.996 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:33:01.031 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:33:01.033 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:33:01.034 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:33:01.034 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:33:01.034 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:33:01.034 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:33:01.034 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:33:01.034 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:33:01.034 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:33:01.034 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:33:01.035 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:33:01.035 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:33:01.035 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:33:01.035 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:33:02.036 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:33:02.036 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:33:03.118 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:33:03.337 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:33:03.337 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:33:03.337 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:33:03.337 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:33:03.337 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:33:04.037 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:33:04.063 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:33:05.889 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:33:06.681 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:33:06.681 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:33:06.682 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:33:06.682 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:33:06.682 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:33:07.292 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:33:09.056 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:33:09.639 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:33:10.188 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:33:10.188 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:33:11.654 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:33:12.270 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:33:12.879 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:33:12.880 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:33:15.063 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:33:15.139 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:33:15.768 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:33:15.768 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:33:16.393 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:33:16.429 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:33:16.459 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:33:16.489 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:33:16.540 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:33:16.583 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:33:16.622 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:33:16.649 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:33:16.676 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:33:16.703 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:33:16.730 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:33:16.731 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:33:16.732 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:33:16.732 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:33:16.732 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:33:16.732 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:33:16.732 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:33:16.732 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:33:16.732 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:33:16.733 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:33:16.733 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:33:16.733 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:33:16.733 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:33:16.733 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:33:17.460 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:33:17.461 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:33:17.909 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:33:17.986 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:33:18.658 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:33:18.659 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:33:19.461 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:33:21.058 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:33:21.473 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:33:22.298 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:33:22.299 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:33:22.588 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:33:23.449 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:33:23.502 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:33:23.541 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:33:23.587 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:33:23.634 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:33:23.675 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:33:23.709 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:33:23.741 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:33:23.773 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:33:23.804 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:33:23.806 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:33:23.806 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:33:23.806 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:33:23.806 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:33:23.806 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:33:23.807 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:33:23.808 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:33:23.808 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:33:23.808 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:33:23.808 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:33:23.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:33:23.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:33:23.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:33:23.874 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:33:24.336 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:33:24.337 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:33:24.398 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:33:25.040 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:33:25.040 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:33:26.337 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:33:26.803 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:33:27.924 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:33:28.512 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:33:28.513 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:33:28.875 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:33:28.896 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:33:28.920 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:33:28.942 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:33:28.968 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:33:28.994 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:33:29.021 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:33:29.047 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:33:29.085 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:33:29.112 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:33:29.140 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:33:29.141 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:33:29.141 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:33:29.141 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:33:29.141 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:33:29.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:33:29.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:33:29.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:33:29.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:33:29.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:33:29.142 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:33:29.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:33:29.143 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:33:29.143 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:33:29.536 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:33:29.860 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:33:29.861 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:33:30.794 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:33:31.510 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:33:31.510 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:33:31.863 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:33:33.067 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:33:34.288 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:33:34.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:33:34.867 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:33:34.867 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:33:35.671 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:33:35.720 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:33:35.761 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:33:35.796 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:33:35.830 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:33:35.863 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:33:35.894 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:33:35.931 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:33:35.957 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:33:35.989 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:33:35.991 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:33:35.991 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:33:35.992 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:33:35.992 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:33:35.992 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:33:35.992 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:33:35.992 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:33:35.992 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:33:35.993 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:33:35.993 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:33:35.993 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:33:35.993 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:33:35.993 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:33:35.997 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:33:36.685 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:33:36.686 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:33:37.257 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:33:38.068 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:33:38.069 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:33:38.687 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:33:39.072 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:33:40.608 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:33:41.205 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:33:41.206 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:33:41.589 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:33:41.612 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:33:41.637 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:33:41.662 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:33:41.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:33:41.716 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:33:41.743 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:33:41.770 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:33:41.797 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:33:41.825 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:33:41.851 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:33:41.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:33:41.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:33:41.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:33:41.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:33:41.853 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:33:41.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:33:41.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:33:41.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:33:41.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:33:41.854 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:33:41.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:33:41.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:33:41.855 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:33:42.059 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:33:42.930 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:33:42.931 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:33:43.602 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:33:44.383 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:33:44.384 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:33:44.931 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:33:45.082 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:33:46.668 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:33:47.382 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:33:47.382 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:33:47.888 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:33:48.087 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:33:48.743 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:33:48.787 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:33:48.818 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:33:48.850 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:33:48.881 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:33:48.908 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:33:48.955 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:33:48.992 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:33:49.032 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:33:49.067 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:33:49.068 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:33:49.069 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:33:49.069 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:33:49.069 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:33:49.069 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:33:49.069 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:33:49.069 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:33:49.069 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:33:49.070 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:33:49.070 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:33:49.070 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:33:49.070 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:33:49.070 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:33:49.607 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:33:49.607 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:33:49.958 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:33:50.433 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:33:50.434 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:33:51.608 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:33:52.131 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:33:52.587 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:33:53.301 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:33:53.301 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:33:54.777 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:33:54.806 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:33:54.833 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:33:54.865 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:33:54.896 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:33:54.909 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:33:54.927 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:33:54.956 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:33:54.979 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:33:55.011 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:33:55.046 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:33:55.074 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:33:55.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:33:55.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:33:55.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:33:55.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:33:55.077 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:33:55.077 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:33:55.077 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:33:55.077 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:33:55.077 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:33:55.077 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:33:55.077 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:33:55.077 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:33:55.078 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:33:55.568 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:33:55.819 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:33:55.819 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:33:56.194 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:33:56.194 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:33:57.820 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:33:58.659 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:33:58.808 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:33:59.429 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:33:59.429 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:34:00.725 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:34:01.583 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:34:01.630 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:34:01.662 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:34:01.673 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:34:01.691 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:34:01.692 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:34:01.719 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:34:01.752 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:34:01.782 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:34:01.810 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:34:01.863 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:34:01.912 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:34:01.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:34:01.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:34:01.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:34:01.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:34:01.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:34:01.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:34:01.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:34:01.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:34:01.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:34:01.915 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:34:01.916 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:34:01.916 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:34:01.916 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:34:02.255 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:34:02.255 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:34:02.739 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:34:02.739 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:34:04.653 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:34:04.740 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:34:05.101 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:34:05.216 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:34:05.216 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:34:07.525 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:34:07.640 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:34:07.791 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:34:08.171 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:34:08.171 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:34:08.381 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:34:08.424 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:34:08.459 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:34:08.494 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:34:08.526 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:34:08.555 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:34:08.584 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:34:08.613 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:34:08.643 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:34:08.670 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:34:08.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:34:08.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:34:08.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:34:08.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:34:08.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:34:08.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:34:08.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:34:08.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:34:08.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:34:08.673 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:34:08.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:34:08.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:34:08.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:34:09.737 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:34:09.737 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:34:10.600 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:34:11.118 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:34:11.119 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:34:11.152 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:34:11.738 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:34:13.709 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:34:13.766 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:34:14.361 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:34:14.395 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:34:14.426 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:34:14.453 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:34:14.457 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:34:14.481 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:34:14.511 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:34:14.538 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:34:14.565 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:34:14.591 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:34:14.619 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:34:14.652 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:34:14.654 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:34:14.654 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:34:14.654 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:34:14.654 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:34:14.654 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:34:14.654 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:34:14.655 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:34:14.655 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:34:14.655 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:34:14.655 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:34:14.655 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:34:14.655 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:34:15.679 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:34:15.679 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:34:17.023 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:34:17.023 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:34:17.023 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:34:17.023 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:34:17.023 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:34:17.184 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:34:17.559 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:34:17.680 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:34:19.906 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:34:19.954 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:34:19.981 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:34:20.015 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:34:20.042 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:34:20.073 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:34:20.099 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:34:20.126 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:34:20.170 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:34:20.200 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:34:20.227 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:34:20.235 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:34:20.253 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:34:20.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:34:20.255 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:34:20.255 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:34:20.255 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:34:20.255 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:34:20.255 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:34:20.255 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:34:20.256 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:34:20.256 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:34:20.256 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:34:20.256 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:34:20.256 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:34:20.256 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:34:21.105 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:34:21.416 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:34:21.416 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:34:22.818 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:34:23.417 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:34:23.491 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:34:24.011 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:34:25.371 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:34:26.066 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:34:26.090 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:34:26.114 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:34:26.141 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:34:26.166 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:34:26.166 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:34:26.166 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:34:26.167 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:34:26.167 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:34:26.169 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:34:26.196 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:34:26.222 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:34:26.268 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:34:26.294 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:34:26.320 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:34:26.362 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:34:26.362 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:34:26.362 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:34:26.362 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:34:26.363 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:34:26.363 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:34:26.363 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:34:26.363 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:34:26.363 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:34:26.363 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:34:26.363 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:34:26.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:34:26.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:34:26.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:34:26.788 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:34:27.330 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:34:27.331 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:34:29.126 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:34:29.275 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:34:29.275 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:34:29.275 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:34:29.275 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:34:29.275 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:34:29.332 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:34:30.081 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:34:31.622 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:34:31.990 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:34:32.011 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:34:32.034 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:34:32.058 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:34:32.086 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:34:32.110 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:34:32.138 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:34:32.164 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:34:32.191 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:34:32.224 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:34:32.249 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:34:32.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:34:32.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:34:32.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:34:32.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:34:32.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:34:32.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:34:32.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:34:32.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:34:32.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:34:32.252 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:34:32.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:34:32.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:34:32.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:34:33.237 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:34:33.358 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:34:33.358 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:34:33.744 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:34:35.172 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:34:35.359 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:34:36.114 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:34:36.726 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:34:37.833 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:34:38.294 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:34:39.154 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:34:39.207 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:34:39.251 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:34:39.290 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:34:39.325 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:34:39.357 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:34:39.388 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:34:39.423 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:34:39.455 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:34:39.490 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:34:39.491 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:34:39.492 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:34:39.492 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:34:39.492 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:34:39.492 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:34:39.492 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:34:39.492 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:34:39.493 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:34:39.493 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:34:39.493 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:34:39.493 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:34:39.493 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:34:39.493 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:34:39.593 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:34:39.593 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:34:39.593 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:34:39.594 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:34:39.594 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:34:40.326 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:34:40.326 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:34:40.539 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:34:41.113 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:34:42.327 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:34:42.692 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:34:42.692 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:34:42.693 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:34:42.693 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:34:43.386 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:34:43.724 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:34:44.603 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:34:44.634 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:34:44.662 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:34:44.688 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:34:44.715 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:34:44.742 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:34:44.768 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:34:44.795 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:34:44.821 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:34:44.877 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:34:44.920 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:34:44.921 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:34:44.921 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:34:44.921 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:34:44.922 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:34:44.922 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:34:44.922 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:34:44.922 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:34:44.922 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:34:44.922 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:34:44.922 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:34:44.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:34:44.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:34:44.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:34:45.551 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:34:45.552 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:34:46.262 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:34:46.904 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:34:47.119 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:34:47.553 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:34:49.573 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:34:49.831 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:34:50.078 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:34:50.101 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:34:50.126 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:34:50.138 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:34:50.177 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:34:50.200 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:34:50.227 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:34:50.254 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:34:50.281 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:34:50.307 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:34:50.335 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:34:50.363 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:34:50.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:34:50.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:34:50.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:34:50.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:34:50.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:34:50.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:34:50.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:34:50.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:34:50.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:34:50.365 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:34:50.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:34:50.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:34:50.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:34:51.145 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:34:51.146 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:34:52.768 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:34:53.112 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:34:53.147 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:34:53.355 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:34:53.356 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:34:55.690 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:34:55.827 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:34:56.293 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:34:56.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:34:56.350 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:34:56.351 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:34:56.362 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:34:56.393 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:34:56.434 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:34:56.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:34:56.474 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:34:56.500 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:34:56.527 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:34:56.555 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:34:56.580 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:34:56.581 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:34:56.581 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:34:56.582 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:34:56.582 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:34:56.582 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:34:56.582 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:34:56.582 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:34:56.582 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:34:56.582 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:34:56.582 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:34:56.583 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:34:56.583 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:34:56.583 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:34:57.284 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:34:57.284 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:34:58.887 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:34:59.120 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:34:59.285 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:34:59.400 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:34:59.400 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:35:01.826 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:35:02.099 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:35:02.938 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:35:02.939 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:35:05.018 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:35:05.131 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:35:05.704 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:35:05.705 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:35:07.758 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:35:07.863 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:35:08.331 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:35:08.332 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:35:10.621 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:35:11.163 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:35:11.163 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:35:11.187 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:35:11.844 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:35:11.876 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:35:11.916 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:35:11.952 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:35:11.984 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:35:12.016 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:35:12.054 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:35:12.081 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:35:12.122 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:35:12.156 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:35:12.199 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:35:12.201 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:35:12.201 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:35:12.201 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:35:12.201 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:35:12.202 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:35:12.202 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:35:12.202 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:35:12.202 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:35:12.202 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:35:12.202 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:35:12.202 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:35:12.203 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:35:12.203 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:35:13.018 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:35:13.019 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:35:13.550 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:35:13.805 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:35:14.078 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:35:14.078 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:35:15.020 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:35:17.010 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:35:17.211 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:35:17.843 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:35:17.867 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:35:17.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:35:17.915 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:35:17.935 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:35:17.936 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:35:17.950 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:35:17.976 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:35:18.003 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:35:18.029 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:35:18.055 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:35:18.092 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:35:18.122 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:35:18.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:35:18.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:35:18.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:35:18.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:35:18.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:35:18.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:35:18.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:35:18.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:35:18.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:35:18.125 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:35:18.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:35:18.126 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:35:18.126 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:35:18.740 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:35:18.740 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:35:19.941 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:35:20.044 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:35:20.590 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:35:20.590 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:35:20.741 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:35:23.327 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:35:23.803 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:35:23.804 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:35:24.000 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:35:24.136 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:35:24.851 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:35:24.892 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:35:24.926 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:35:24.979 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:35:25.024 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:35:25.061 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:35:25.097 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:35:25.131 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:35:25.177 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:35:25.212 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:35:25.214 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:35:25.214 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:35:25.214 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:35:25.214 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:35:25.214 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:35:25.214 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:35:25.214 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:35:25.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:35:25.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:35:25.215 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:35:25.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:35:25.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:35:25.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:35:25.805 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:35:25.806 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:35:26.015 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:35:26.549 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:35:26.550 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:35:26.825 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:35:27.806 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:35:29.132 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:35:29.943 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:35:29.943 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:35:30.137 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:35:30.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:35:30.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:35:30.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:35:30.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:35:30.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:35:30.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:35:30.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:35:30.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:35:30.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:35:30.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:35:30.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:35:30.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:35:30.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:35:30.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:35:30.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:35:30.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:35:30.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:35:30.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:35:30.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:35:30.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:35:30.849 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:35:30.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:35:30.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:35:30.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:35:31.425 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:35:31.425 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:35:32.422 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:35:32.921 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:35:33.001 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:35:33.002 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:35:33.426 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:35:36.078 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:35:36.620 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:35:36.624 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:35:36.624 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:35:36.652 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:35:36.682 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:35:36.708 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:35:36.734 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:35:36.761 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:35:36.787 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:35:36.813 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:35:36.824 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:35:36.840 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:35:36.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:35:36.895 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:35:36.896 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:35:36.896 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:35:36.896 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:35:36.896 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:35:36.897 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:35:36.897 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:35:36.897 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:35:36.897 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:35:36.897 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:35:36.897 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:35:36.897 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:35:36.898 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:35:36.898 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:35:37.639 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:35:37.639 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:35:38.982 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:35:39.507 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:35:39.507 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:35:39.640 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:35:39.804 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:35:42.275 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:35:42.973 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:35:42.998 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:35:43.031 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:35:43.048 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:35:43.049 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:35:43.071 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:35:43.107 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:35:43.135 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:35:43.162 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:35:43.189 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:35:43.215 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:35:43.242 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:35:43.268 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:35:43.269 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:35:43.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:35:43.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:35:43.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:35:43.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:35:43.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:35:43.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:35:43.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:35:43.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:35:43.271 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:35:43.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:35:43.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:35:43.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:35:43.914 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:35:43.915 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:35:44.152 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:35:45.489 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:35:45.916 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:35:46.013 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:35:46.014 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:35:46.871 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:35:48.608 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:35:49.004 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:35:49.461 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:35:49.510 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:35:49.510 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:35:49.516 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:35:49.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:35:49.592 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:35:49.622 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:35:49.653 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:35:49.680 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:35:49.707 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:35:49.734 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:35:49.763 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:35:49.764 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:35:49.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:35:49.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:35:49.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:35:49.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:35:49.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:35:49.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:35:49.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:35:49.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:35:49.766 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:35:49.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:35:49.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:35:49.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:35:50.416 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:35:50.417 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:35:50.642 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:35:52.061 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:35:52.418 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:35:52.590 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:35:52.590 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:35:53.276 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:35:55.038 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:35:55.060 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:35:55.083 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:35:55.106 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:35:55.132 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:35:55.158 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:35:55.184 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:35:55.210 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:35:55.238 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:35:55.265 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:35:55.291 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:35:55.293 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:35:55.293 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:35:55.293 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:35:55.293 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:35:55.293 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:35:55.293 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:35:55.294 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:35:55.294 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:35:55.294 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:35:55.294 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:35:55.294 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:35:55.294 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:35:55.294 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:35:55.822 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:35:55.882 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:35:55.882 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:35:56.472 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:35:56.473 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:35:57.168 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:35:57.885 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:35:58.631 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:35:59.414 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:36:00.063 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:36:00.211 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:36:00.236 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:36:00.265 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:36:00.285 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:36:00.311 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:36:00.337 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:36:00.362 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:36:00.389 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:36:00.417 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:36:00.445 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:36:00.471 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:36:00.473 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:36:00.473 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:36:00.474 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:36:00.474 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:36:00.474 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:36:00.475 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:36:00.475 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:36:00.475 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:36:00.475 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:36:00.475 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:36:00.476 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:36:00.476 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:36:00.476 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:36:01.314 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:36:01.315 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:36:02.384 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:36:02.385 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:36:02.385 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:36:02.385 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:36:02.385 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:36:02.959 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:36:03.135 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:36:03.316 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:36:05.684 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:36:05.684 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:36:05.684 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:36:05.684 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:36:05.684 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:36:05.827 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:36:05.967 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:36:05.989 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:36:06.015 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:36:06.042 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:36:06.065 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:36:06.092 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:36:06.119 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:36:06.156 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:36:06.192 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:36:06.232 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:36:06.268 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:36:06.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:36:06.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:36:06.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:36:06.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:36:06.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:36:06.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:36:06.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:36:06.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:36:06.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:36:06.271 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:36:06.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:36:06.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:36:06.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:36:06.450 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:36:07.073 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:36:07.073 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:36:09.074 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:36:09.163 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:36:09.235 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:36:10.126 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:36:11.860 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:36:12.105 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:36:12.127 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:36:12.152 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:36:12.185 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:36:12.217 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:36:12.248 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:36:12.276 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:36:12.305 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:36:12.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:36:12.361 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:36:12.387 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:36:12.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:36:12.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:36:12.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:36:12.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:36:12.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:36:12.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:36:12.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:36:12.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:36:12.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:36:12.391 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:36:12.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:36:12.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:36:12.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:36:12.601 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:36:12.601 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:36:12.601 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:36:12.601 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:36:12.602 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:36:13.286 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:36:13.287 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:36:13.683 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:36:15.195 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:36:15.287 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:36:15.857 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:36:15.857 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:36:15.857 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:36:15.857 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:36:15.858 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:36:16.518 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:36:17.911 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:36:18.251 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:36:18.274 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:36:18.297 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:36:18.319 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:36:18.378 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:36:18.417 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:36:18.445 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:36:18.496 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:36:18.548 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:36:18.579 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:36:18.609 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:36:18.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:36:18.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:36:18.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:36:18.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:36:18.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:36:18.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:36:18.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:36:18.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:36:18.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:36:18.613 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:36:18.614 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:36:18.614 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:36:18.614 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:36:19.340 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:36:19.340 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:36:19.738 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:36:20.423 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:36:21.174 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:36:21.341 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:36:22.839 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:36:23.560 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:36:24.115 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:36:24.149 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:36:24.182 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:36:24.213 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:36:24.235 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:36:24.238 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:36:24.266 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:36:24.292 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:36:24.323 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:36:24.350 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:36:24.378 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:36:24.406 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:36:24.408 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:36:24.408 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:36:24.408 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:36:24.408 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:36:24.408 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:36:24.408 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:36:24.409 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:36:24.409 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:36:24.409 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:36:24.409 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:36:24.409 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:36:24.409 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:36:24.410 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:36:25.397 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:36:25.397 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:36:26.070 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:36:26.070 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:36:26.070 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:36:26.071 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:36:26.071 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:36:26.718 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:36:27.398 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:36:27.469 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:36:29.218 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:36:29.219 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:36:29.219 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:36:29.219 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:36:29.996 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:36:30.254 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:36:30.415 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:36:30.439 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:36:30.466 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:36:30.490 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:36:30.517 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:36:30.545 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:36:30.570 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:36:30.599 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:36:30.631 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:36:30.656 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:36:30.683 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:36:30.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:36:30.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:36:30.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:36:30.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:36:30.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:36:30.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:36:30.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:36:30.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:36:30.686 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:36:30.686 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:36:30.686 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:36:30.686 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:36:30.687 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:36:31.638 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:36:31.638 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:36:32.722 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:36:33.278 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:36:33.639 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:36:34.197 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:36:36.009 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:36:36.518 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:36:36.526 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:36:37.141 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:36:37.372 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:36:37.397 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:36:37.434 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:36:37.468 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:36:37.498 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:36:37.530 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:36:37.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:36:37.583 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:36:37.631 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:36:37.663 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:36:37.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:36:37.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:36:37.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:36:37.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:36:37.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:36:37.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:36:37.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:36:37.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:36:37.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:36:37.667 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:36:37.667 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:36:37.667 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:36:37.667 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:36:38.476 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:36:38.476 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:36:39.333 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:36:39.837 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:36:39.837 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:36:40.179 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:36:40.478 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:36:42.209 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:36:42.760 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:36:42.760 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:36:42.761 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:36:43.170 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:36:43.198 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:36:43.223 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:36:43.246 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:36:43.272 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:36:43.298 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:36:43.325 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:36:43.352 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:36:43.378 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:36:43.406 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:36:43.457 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:36:43.457 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:36:43.457 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:36:43.457 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:36:43.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:36:43.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:36:43.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:36:43.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:36:43.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:36:43.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:36:43.458 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:36:43.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:36:43.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:36:43.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:36:44.134 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:36:44.135 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:36:45.353 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:36:45.911 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:36:45.912 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:36:46.136 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:36:46.850 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:36:48.481 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:36:49.005 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:36:49.036 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:36:49.046 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:36:49.047 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:36:49.061 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:36:49.086 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:36:49.113 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:36:49.138 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:36:49.170 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:36:49.196 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:36:49.223 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:36:49.249 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:36:49.275 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:36:49.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:36:49.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:36:49.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:36:49.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:36:49.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:36:49.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:36:49.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:36:49.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:36:49.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:36:49.278 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:36:49.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:36:49.278 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:36:49.279 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:36:49.777 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:36:49.925 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:36:49.925 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:36:51.552 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:36:51.927 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:36:52.105 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:36:52.106 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:36:53.212 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:36:54.724 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:36:54.747 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:36:54.762 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:36:54.770 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:36:54.794 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:36:54.819 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:36:54.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:36:54.873 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:36:54.901 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:36:54.940 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:36:54.966 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:36:54.999 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:36:55.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:36:55.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:36:55.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:36:55.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:36:55.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:36:55.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:36:55.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:36:55.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:36:55.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:36:55.002 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:36:55.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:36:55.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:36:55.003 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:36:55.306 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:36:55.306 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:36:55.753 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:36:55.753 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:36:56.214 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:36:57.555 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:36:57.754 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:36:58.167 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:36:58.167 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:36:59.191 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:37:00.725 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:37:00.759 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:37:00.790 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:37:00.818 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:37:00.848 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:37:00.874 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:37:00.889 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:37:00.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:37:00.931 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:37:00.957 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:37:00.984 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:37:01.012 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:37:01.013 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:37:01.013 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:37:01.013 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:37:01.013 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:37:01.013 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:37:01.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:37:01.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:37:01.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:37:01.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:37:01.014 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:37:01.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:37:01.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:37:01.015 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:37:01.548 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:37:01.548 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:37:01.747 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:37:01.747 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:37:02.004 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:37:03.634 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:37:03.748 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:37:04.251 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:37:04.252 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:37:05.207 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:37:06.850 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:37:06.877 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:37:06.902 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:37:06.925 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:37:06.951 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:37:06.978 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:37:07.003 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:37:07.029 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:37:07.055 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:37:07.068 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:37:07.082 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:37:07.108 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:37:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:37:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:37:07.109 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:37:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:37:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:37:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:37:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:37:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:37:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:37:07.110 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:37:07.111 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:37:07.111 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:37:07.111 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:37:07.680 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:37:07.681 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:37:07.848 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:37:07.848 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:37:08.254 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:37:09.850 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:37:09.877 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:37:10.708 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:37:10.709 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:37:11.357 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:37:13.138 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:37:13.163 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:37:13.200 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:37:13.231 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:37:13.263 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:37:13.295 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:37:13.324 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:37:13.350 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:37:13.355 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:37:13.376 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:37:13.403 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:37:13.433 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:37:13.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:37:13.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:37:13.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:37:13.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:37:13.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:37:13.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:37:13.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:37:13.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:37:13.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:37:13.435 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:37:13.435 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:37:13.435 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:37:13.435 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:37:14.016 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:37:14.016 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:37:14.185 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:37:14.185 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:37:14.379 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:37:16.186 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:37:16.357 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:37:17.070 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:37:17.070 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:37:17.390 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:37:18.898 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:37:18.907 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:37:18.929 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:37:18.953 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:37:18.979 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:37:19.005 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:37:19.055 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:37:19.098 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:37:19.136 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:37:19.172 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:37:19.206 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:37:19.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:37:19.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:37:19.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:37:19.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:37:19.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:37:19.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:37:19.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:37:19.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:37:19.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:37:19.210 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:37:19.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:37:19.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:37:19.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:37:19.827 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:37:19.828 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:37:20.099 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:37:20.220 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:37:20.647 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:37:20.648 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:37:21.829 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:37:23.174 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:37:23.416 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:37:24.115 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:37:24.116 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:37:24.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:37:25.709 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:37:25.740 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:37:25.776 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:37:25.806 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:37:25.835 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:37:25.862 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:37:25.889 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:37:25.915 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:37:25.944 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:37:25.981 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:37:25.982 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:37:25.982 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:37:25.982 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:37:25.983 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:37:25.983 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:37:25.983 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:37:25.983 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:37:25.983 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:37:25.983 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:37:25.983 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:37:25.984 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:37:25.984 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:37:25.984 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:37:26.209 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:37:26.368 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:37:26.539 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:37:26.539 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:37:26.881 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:37:26.882 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:37:28.540 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:37:29.201 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:37:29.219 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:37:30.249 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:37:30.249 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:37:31.085 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:37:31.114 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:37:31.142 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:37:31.168 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:37:31.195 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:37:31.220 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:37:31.250 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:37:31.280 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:37:31.307 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:37:31.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:37:31.359 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:37:31.360 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:37:31.360 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:37:31.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:37:31.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:37:31.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:37:31.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:37:31.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:37:31.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:37:31.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:37:31.362 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:37:31.362 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:37:31.362 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:37:31.362 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:37:31.957 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:37:32.144 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:37:32.144 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:37:32.672 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:37:33.484 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:37:33.485 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:37:34.145 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:37:35.235 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:37:35.897 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:37:36.552 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:37:36.552 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:37:37.061 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:37:37.093 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:37:37.124 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:37:37.153 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:37:37.182 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:37:37.209 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:37:37.235 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:37:37.264 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:37:37.290 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:37:37.318 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:37:37.345 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:37:37.347 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:37:37.347 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:37:37.347 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:37:37.347 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:37:37.347 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:37:37.348 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:37:37.348 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:37:37.348 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:37:37.348 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:37:37.348 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:37:37.348 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:37:37.348 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:37:37.349 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:37:38.072 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:37:38.185 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:37:38.185 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:37:38.717 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:37:39.287 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:37:39.287 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:37:40.186 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:37:41.216 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:37:41.842 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:37:42.643 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:37:42.644 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:37:43.818 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:37:44.654 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:37:44.655 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:37:44.655 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:37:44.655 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:37:45.266 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:37:46.858 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:37:47.818 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:37:47.819 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:37:47.819 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:37:47.819 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:37:47.819 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:37:48.364 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:37:49.497 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:37:50.638 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:37:50.638 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:37:50.638 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:37:50.638 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:37:50.638 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:37:51.349 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:37:52.239 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:37:53.123 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:37:53.158 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:37:53.191 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:37:53.222 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:37:53.248 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:37:53.284 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:37:53.316 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:37:53.347 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:37:53.375 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:37:53.402 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:37:53.429 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:37:53.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:37:53.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:37:53.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:37:53.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:37:53.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:37:53.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:37:53.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:37:53.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:37:53.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:37:53.433 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:37:53.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:37:53.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:37:53.434 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:37:53.553 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:37:53.935 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:37:53.936 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:37:54.294 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:37:55.025 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:37:55.937 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:37:56.438 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:37:57.299 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:37:59.052 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:37:59.076 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:37:59.099 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:37:59.129 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:37:59.160 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:37:59.190 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:37:59.219 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:37:59.224 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:37:59.251 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:37:59.291 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:37:59.320 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:37:59.349 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:37:59.354 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:37:59.354 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:37:59.355 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:37:59.355 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:37:59.355 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:37:59.355 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:37:59.355 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:37:59.355 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:37:59.355 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:37:59.356 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:37:59.356 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:37:59.356 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:37:59.356 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:37:59.926 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:37:59.926 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:37:59.926 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:37:59.926 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:37:59.926 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:38:00.176 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:38:00.176 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:38:00.510 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:38:01.816 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:38:02.177 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:38:02.529 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:38:02.529 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:38:02.529 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:38:02.529 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:38:03.372 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:38:05.079 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:38:05.113 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:38:05.139 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:38:05.171 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:38:05.201 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:38:05.229 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:38:05.241 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:38:05.256 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:38:05.282 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:38:05.308 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:38:05.335 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:38:05.362 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:38:05.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:38:05.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:38:05.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:38:05.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:38:05.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:38:05.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:38:05.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:38:05.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:38:05.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:38:05.365 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:38:05.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:38:05.365 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:38:05.366 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:38:06.027 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:38:06.200 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:38:06.200 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:38:06.551 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:38:07.995 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:38:08.202 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:38:08.520 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:38:09.195 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:38:11.411 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:38:11.435 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:38:11.459 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:38:11.484 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:38:11.512 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:38:11.542 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:38:11.571 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:38:11.598 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:38:11.624 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:38:11.651 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:38:11.677 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:38:11.678 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:38:11.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:38:11.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:38:11.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:38:11.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:38:11.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:38:11.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:38:11.680 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:38:11.680 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:38:11.680 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:38:11.680 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:38:11.680 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:38:11.680 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:38:11.712 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:38:11.712 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:38:11.712 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:38:11.712 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:38:11.861 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:38:12.333 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:38:12.334 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:38:12.593 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:38:14.335 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:38:14.508 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:38:14.508 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:38:14.508 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:38:14.508 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:38:14.508 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:38:14.775 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:38:15.185 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:38:17.353 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:38:18.188 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:38:18.188 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:38:18.188 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:38:18.188 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:38:18.188 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:38:18.216 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:38:18.226 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:38:18.244 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:38:18.270 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:38:18.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:38:18.322 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:38:18.348 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:38:18.376 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:38:18.403 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:38:18.429 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:38:18.454 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:38:18.456 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:38:18.456 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:38:18.456 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:38:18.456 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:38:18.457 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:38:18.457 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:38:18.457 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:38:18.457 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:38:18.457 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:38:18.457 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:38:18.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:38:18.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:38:18.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:38:18.901 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:38:19.310 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:38:19.311 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:38:20.913 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:38:21.155 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:38:21.312 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:38:21.476 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:38:21.476 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:38:24.340 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:38:24.363 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:38:24.386 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:38:24.409 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:38:24.435 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:38:24.462 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:38:24.488 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:38:24.514 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:38:24.534 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:38:24.547 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:38:24.574 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:38:24.600 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:38:24.601 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:38:24.601 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:38:24.602 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:38:24.602 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:38:24.602 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:38:24.602 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:38:24.602 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:38:24.602 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:38:24.603 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:38:24.603 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:38:24.603 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:38:24.603 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:38:24.603 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:38:25.220 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:38:25.220 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:38:25.245 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:38:25.392 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:38:25.393 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:38:27.329 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:38:27.394 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:38:27.862 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:38:27.915 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:38:27.916 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:38:30.388 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:38:30.410 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:38:30.439 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:38:30.463 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:38:30.492 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:38:30.514 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:38:30.540 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:38:30.566 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:38:30.592 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:38:30.620 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:38:30.644 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:38:30.646 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:38:30.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:38:30.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:38:30.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:38:30.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:38:30.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:38:30.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:38:30.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:38:30.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:38:30.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:38:30.649 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:38:30.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:38:30.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:38:30.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:38:31.325 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:38:31.325 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:38:31.509 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:38:31.510 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:38:32.265 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:38:33.326 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:38:33.642 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:38:34.415 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:38:34.415 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:38:35.161 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:38:35.569 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:38:36.420 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:38:36.449 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:38:36.491 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:38:36.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:38:36.567 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:38:36.598 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:38:36.627 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:38:36.657 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:38:36.688 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:38:36.739 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:38:36.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:38:36.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:38:36.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:38:36.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:38:36.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:38:36.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:38:36.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:38:36.743 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:38:36.743 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:38:36.743 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:38:36.743 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:38:36.743 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:38:36.743 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:38:36.824 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:38:37.438 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:38:37.439 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:38:37.743 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:38:37.744 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:38:39.250 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:38:39.440 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:38:39.987 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:38:40.682 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:38:40.682 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:38:41.925 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:38:41.939 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:38:41.949 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:38:41.972 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:38:41.995 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:38:42.024 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:38:42.052 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:38:42.079 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:38:42.107 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:38:42.133 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:38:42.160 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:38:42.188 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:38:42.189 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:38:42.189 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:38:42.189 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:38:42.189 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:38:42.189 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:38:42.190 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:38:42.190 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:38:42.190 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:38:42.190 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:38:42.190 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:38:42.191 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:38:42.191 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:38:42.191 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:38:42.945 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:38:42.946 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:38:43.694 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:38:44.263 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:38:44.264 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:38:44.947 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:38:45.263 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:38:46.953 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:38:47.631 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:38:47.659 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:38:47.672 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:38:47.673 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:38:47.697 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:38:47.730 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:38:47.767 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:38:47.796 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:38:47.822 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:38:47.848 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:38:47.875 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:38:47.902 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:38:47.928 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:38:47.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:38:47.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:38:47.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:38:47.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:38:47.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:38:47.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:38:47.931 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:38:47.931 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:38:47.931 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:38:47.931 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:38:47.931 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:38:47.931 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:38:47.931 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:38:48.184 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:38:48.543 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:38:48.543 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:38:50.173 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:38:50.545 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:38:50.726 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:38:50.727 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:38:51.264 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:38:53.200 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:38:53.232 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:38:53.263 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:38:53.288 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:38:53.314 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:38:53.340 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:38:53.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:38:53.391 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:38:53.415 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:38:53.424 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:38:53.451 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:38:53.477 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:38:53.480 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:38:53.480 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:38:53.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:38:53.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:38:53.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:38:53.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:38:53.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:38:53.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:38:53.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:38:53.482 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:38:53.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:38:53.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:38:53.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:38:54.133 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:38:54.134 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:38:54.151 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:38:54.278 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:38:54.278 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:38:56.279 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:38:56.343 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:38:56.886 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:38:56.887 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:38:57.074 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:38:59.260 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:38:59.290 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:38:59.315 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:38:59.340 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:38:59.367 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:38:59.393 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:38:59.419 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:38:59.430 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:38:59.446 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:38:59.494 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:38:59.533 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:38:59.568 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:38:59.570 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:38:59.570 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:38:59.571 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:38:59.571 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:38:59.571 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:38:59.571 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:38:59.571 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:38:59.571 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:38:59.571 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:38:59.572 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:38:59.572 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:38:59.572 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:38:59.572 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:38:59.944 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:38:59.945 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:39:00.040 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:39:00.343 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:39:00.343 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:39:02.345 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:39:02.468 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:39:03.128 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:39:03.129 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:39:03.263 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:39:05.290 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:39:05.316 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:39:05.342 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:39:05.367 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:39:05.393 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:39:05.420 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:39:05.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:39:05.474 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:39:05.517 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:39:05.547 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:39:05.574 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:39:05.575 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:39:05.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:39:05.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:39:05.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:39:05.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:39:05.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:39:05.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:39:05.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:39:05.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:39:05.577 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:39:05.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:39:05.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:39:05.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:39:05.821 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:39:05.934 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:39:06.091 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:39:06.091 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:39:06.461 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:39:06.462 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:39:08.092 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:39:08.856 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:39:09.424 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:39:09.425 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:39:10.575 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:39:10.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:39:10.653 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:39:10.685 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:39:10.714 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:39:10.744 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:39:10.772 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:39:10.798 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:39:10.826 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:39:10.855 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:39:10.882 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:39:10.884 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:39:10.884 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:39:10.884 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:39:10.884 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:39:10.884 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:39:10.885 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:39:10.885 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:39:10.885 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:39:10.885 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:39:10.885 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:39:10.885 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:39:10.885 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:39:10.886 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:39:11.588 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:39:11.588 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:39:11.837 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:39:12.346 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:39:12.346 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:39:13.588 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:39:14.201 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:39:14.852 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:39:14.852 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:39:16.060 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:39:16.912 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:39:16.956 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:39:16.994 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:39:17.026 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:39:17.046 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:39:17.055 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:39:17.082 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:39:17.111 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:39:17.137 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:39:17.165 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:39:17.191 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:39:17.193 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:39:17.193 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:39:17.193 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:39:17.193 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:39:17.193 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:39:17.193 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:39:17.194 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:39:17.194 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:39:17.194 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:39:17.194 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:39:17.194 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:39:17.194 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:39:17.194 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:39:17.752 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:39:17.753 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:39:17.921 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:39:17.921 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:39:19.271 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:39:19.766 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:39:19.922 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:39:20.323 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:39:20.324 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:39:21.916 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:39:22.359 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:39:22.445 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:39:23.186 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:39:23.186 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:39:23.214 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:39:23.248 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:39:23.289 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:39:23.321 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:39:23.349 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:39:23.377 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:39:23.405 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:39:23.433 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:39:23.459 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:39:23.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:39:23.494 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:39:23.494 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:39:23.494 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:39:23.494 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:39:23.494 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:39:23.494 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:39:23.495 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:39:23.495 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:39:23.495 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:39:23.495 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:39:23.495 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:39:23.495 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:39:23.495 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:39:24.140 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:39:24.141 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:39:25.282 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:39:25.929 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:39:25.929 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:39:25.929 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:39:25.929 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:39:26.142 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:39:26.392 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:39:27.871 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:39:28.580 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:39:28.610 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:39:28.636 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:39:28.659 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:39:28.686 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:39:28.715 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:39:28.741 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:39:28.774 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:39:28.808 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:39:28.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:39:28.876 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:39:28.880 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:39:28.880 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:39:28.880 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:39:28.881 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:39:28.881 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:39:28.881 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:39:28.881 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:39:28.881 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:39:28.881 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:39:28.881 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:39:28.882 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:39:28.882 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:39:28.882 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:39:28.986 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:39:29.764 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:39:29.764 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:39:29.987 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:39:31.307 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:39:31.765 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:39:32.330 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:39:32.945 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:39:33.923 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:39:34.282 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:39:34.305 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:39:34.329 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:39:34.352 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:39:34.383 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:39:34.413 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:39:34.442 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:39:34.472 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:39:34.502 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:39:34.529 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:39:34.570 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:39:34.571 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:39:34.571 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:39:34.571 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:39:34.572 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:39:34.572 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:39:34.572 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:39:34.572 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:39:34.572 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:39:34.572 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:39:34.573 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:39:34.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:39:34.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:39:34.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:39:35.333 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:39:35.333 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:39:35.780 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:39:35.780 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:39:35.780 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:39:35.781 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:39:36.357 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:39:36.876 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:39:37.334 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:39:38.584 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:39:38.584 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:39:38.584 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:39:38.584 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:39:38.584 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:39:39.312 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:39:39.527 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:39:39.871 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:39:40.735 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:39:40.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:39:40.813 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:39:40.848 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:39:40.880 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:39:40.912 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:39:40.946 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:39:40.986 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:39:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:39:41.078 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:39:41.080 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:39:41.080 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:39:41.081 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:39:41.081 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:39:41.081 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:39:41.081 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:39:41.081 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:39:41.082 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:39:41.082 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:39:41.082 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:39:41.082 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:39:41.082 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:39:41.083 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:39:41.856 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:39:41.856 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:39:42.245 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:39:42.335 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:39:42.883 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:39:43.860 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:39:44.995 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:39:45.048 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:39:45.879 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:39:46.458 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:39:46.480 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:39:46.506 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:39:46.528 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:39:46.555 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:39:46.586 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:39:46.612 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:39:46.638 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:39:46.665 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:39:46.695 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:39:46.721 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:39:46.723 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:39:46.723 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:39:46.723 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:39:46.723 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:39:46.723 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:39:46.724 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:39:46.724 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:39:46.724 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:39:46.724 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:39:46.724 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:39:46.725 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:39:46.725 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:39:46.725 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:39:47.503 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:39:47.503 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:39:48.521 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:39:48.521 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:39:48.521 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:39:48.521 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:39:48.521 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:39:49.068 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:39:49.285 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:39:49.504 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:39:51.748 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:39:51.748 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:39:51.748 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:39:51.748 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:39:51.748 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:39:51.949 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:39:52.260 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:39:52.284 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:39:52.307 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:39:52.313 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:39:52.338 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:39:52.364 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:39:52.390 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:39:52.418 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:39:52.443 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:39:52.469 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:39:52.496 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:39:52.524 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:39:52.525 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:39:52.526 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:39:52.526 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:39:52.526 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:39:52.526 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:39:52.526 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:39:52.527 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:39:52.527 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:39:52.527 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:39:52.527 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:39:52.527 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:39:52.528 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:39:52.528 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:39:53.241 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:39:53.241 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:39:55.018 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:39:55.242 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:39:55.542 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:39:55.572 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:39:58.221 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:39:58.305 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:39:58.797 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:40:01.193 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:40:01.298 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:40:01.882 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:40:03.990 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:40:04.039 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:40:04.670 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:40:04.671 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:40:06.819 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:40:06.885 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:40:07.441 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:40:07.441 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:40:07.955 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:40:07.991 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:40:08.029 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:40:08.062 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:40:08.096 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:40:08.130 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:40:08.164 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:40:08.194 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:40:08.224 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:40:08.254 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:40:08.283 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:40:08.287 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:40:08.287 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:40:08.287 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:40:08.288 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:40:08.288 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:40:08.288 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:40:08.288 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:40:08.289 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:40:08.289 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:40:08.289 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:40:08.289 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:40:08.290 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:40:08.290 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:40:09.075 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:40:09.076 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:40:09.593 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:40:09.621 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:40:10.167 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:40:10.167 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:40:11.077 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:40:12.949 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:40:13.335 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:40:13.475 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:40:13.475 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:40:14.326 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:40:14.358 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:40:14.386 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:40:14.412 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:40:14.440 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:40:14.467 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:40:14.497 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:40:14.523 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:40:14.549 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:40:14.586 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:40:14.631 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:40:14.632 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:40:14.632 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:40:14.632 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:40:14.632 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:40:14.632 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:40:14.633 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:40:14.633 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:40:14.633 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:40:14.633 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:40:14.633 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:40:14.633 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:40:14.633 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:40:14.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:40:15.505 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:40:15.505 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:40:15.940 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:40:16.016 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:40:16.568 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:40:16.569 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:40:17.506 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:40:19.314 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:40:19.452 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:40:20.283 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:40:20.283 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:40:20.664 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:40:20.687 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:40:20.710 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:40:20.738 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:40:20.769 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:40:20.798 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:40:20.824 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:40:20.851 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:40:20.878 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:40:20.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:40:20.935 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:40:20.936 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:40:20.937 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:40:20.937 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:40:20.937 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:40:20.937 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:40:20.937 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:40:20.938 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:40:20.938 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:40:20.938 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:40:20.938 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:40:20.938 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:40:20.938 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:40:20.938 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:40:21.673 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:40:21.673 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:40:21.959 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:40:22.357 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:40:23.067 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:40:23.068 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:40:23.675 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:40:25.322 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:40:25.490 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:40:26.380 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:40:26.381 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:40:26.766 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:40:26.787 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:40:26.810 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:40:26.832 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:40:26.858 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:40:26.884 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:40:26.910 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:40:26.937 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:40:26.964 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:40:27.005 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:40:27.042 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:40:27.044 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:40:27.044 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:40:27.045 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:40:27.045 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:40:27.045 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:40:27.045 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:40:27.045 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:40:27.045 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:40:27.045 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:40:27.046 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:40:27.046 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:40:27.046 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:40:27.046 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:40:27.797 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:40:27.797 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:40:28.115 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:40:28.511 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:40:29.234 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:40:29.235 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:40:29.798 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:40:30.909 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:40:31.578 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:40:32.303 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:40:32.304 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:40:32.709 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:40:32.739 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:40:32.768 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:40:32.798 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:40:32.827 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:40:32.856 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:40:32.889 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:40:32.915 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:40:32.945 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:40:32.972 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:40:32.998 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:40:32.999 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:40:32.999 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:40:32.999 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:40:33.000 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:40:33.000 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:40:33.000 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:40:33.000 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:40:33.000 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:40:33.000 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:40:33.000 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:40:33.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:40:33.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:40:33.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:40:33.869 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:40:33.870 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:40:34.014 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:40:34.372 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:40:35.045 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:40:35.045 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:40:35.870 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:40:36.932 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:40:37.167 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:40:38.173 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:40:38.173 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:40:38.607 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:40:38.634 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:40:38.662 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:40:38.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:40:38.716 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:40:38.743 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:40:38.771 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:40:38.801 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:40:38.836 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:40:38.867 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:40:38.912 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:40:38.913 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:40:38.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:40:38.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:40:38.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:40:38.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:40:38.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:40:38.914 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:40:38.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:40:38.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:40:38.915 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:40:38.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:40:38.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:40:38.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:40:39.637 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:40:39.637 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:40:39.940 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:40:40.349 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:40:41.236 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:40:41.236 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:40:41.638 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:40:42.319 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:40:43.980 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:40:44.225 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:40:44.261 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:40:44.293 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:40:44.325 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:40:44.356 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:40:44.385 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:40:44.411 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:40:44.438 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:40:44.464 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:40:44.494 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:40:44.494 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:40:44.495 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:40:44.520 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:40:44.522 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:40:44.522 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:40:44.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:40:44.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:40:44.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:40:44.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:40:44.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:40:44.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:40:44.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:40:44.523 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:40:44.524 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:40:44.524 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:40:44.524 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:40:44.973 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:40:44.973 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:40:45.584 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:40:46.974 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:40:46.994 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:40:47.739 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:40:47.740 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:40:49.572 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:40:49.631 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:40:49.660 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:40:49.688 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:40:49.716 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:40:49.742 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:40:49.768 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:40:49.794 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:40:49.820 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:40:49.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:40:49.873 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:40:49.875 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:40:49.875 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:40:49.876 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:40:49.876 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:40:49.876 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:40:49.876 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:40:49.876 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:40:49.876 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:40:49.877 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:40:49.877 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:40:49.877 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:40:49.877 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:40:49.877 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:40:50.331 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:40:50.399 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:40:50.399 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:40:50.788 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:40:50.789 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:40:52.400 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:40:52.735 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:40:53.362 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:40:53.363 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:40:55.033 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:40:55.066 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:40:55.098 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:40:55.124 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:40:55.152 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:40:55.178 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:40:55.204 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:40:55.248 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:40:55.288 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:40:55.325 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:40:55.360 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:40:55.362 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:40:55.362 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:40:55.363 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:40:55.363 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:40:55.363 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:40:55.363 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:40:55.363 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:40:55.363 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:40:55.363 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:40:55.363 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:40:55.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:40:55.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:40:55.364 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:40:55.921 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:40:55.921 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:40:55.956 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:40:56.434 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:40:56.434 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:40:57.923 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:40:58.556 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:40:58.912 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:40:59.159 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:40:59.160 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:41:00.841 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:41:00.864 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:41:00.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:41:00.913 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:41:00.940 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:41:00.970 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:41:00.997 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:41:01.034 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:41:01.062 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:41:01.090 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:41:01.117 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:41:01.119 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:41:01.119 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:41:01.119 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:41:01.120 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:41:01.120 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:41:01.120 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:41:01.120 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:41:01.120 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:41:01.120 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:41:01.120 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:41:01.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:41:01.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:41:01.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:41:01.739 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:41:01.866 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:41:01.866 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:41:02.008 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:41:02.231 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:41:02.231 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:41:03.867 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:41:04.581 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:41:05.235 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:41:05.236 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:41:05.342 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:41:06.853 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:41:06.880 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:41:06.904 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:41:06.928 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:41:06.953 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:41:06.979 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:41:07.006 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:41:07.033 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:41:07.059 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:41:07.086 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:41:07.112 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:41:07.114 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:41:07.114 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:41:07.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:41:07.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:41:07.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:41:07.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:41:07.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:41:07.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:41:07.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:41:07.115 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:41:07.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:41:07.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:41:07.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:41:07.686 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:41:07.687 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:41:08.241 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:41:08.242 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:41:08.242 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:41:08.242 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:41:08.259 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:41:08.909 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:41:09.687 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:41:12.024 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:41:12.024 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:41:12.024 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:41:12.025 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:41:12.025 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:41:12.244 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:41:12.268 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:41:12.290 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:41:12.313 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:41:12.338 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:41:12.364 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:41:12.391 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:41:12.401 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:41:12.420 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:41:12.445 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:41:12.472 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:41:12.497 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:41:12.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:41:12.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:41:12.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:41:12.500 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:41:12.500 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:41:12.500 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:41:12.500 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:41:12.500 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:41:12.501 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:41:12.501 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:41:12.501 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:41:12.501 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:41:12.501 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:41:12.629 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:41:13.448 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:41:13.449 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:41:14.925 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:41:15.143 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:41:15.450 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:41:15.503 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:41:18.426 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:41:18.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:41:18.972 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:41:19.368 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:41:19.407 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:41:19.442 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:41:19.443 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:41:19.489 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:41:19.529 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:41:19.560 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:41:19.588 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:41:19.616 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:41:19.645 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:41:19.671 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:41:19.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:41:19.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:41:19.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:41:19.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:41:19.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:41:19.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:41:19.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:41:19.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:41:19.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:41:19.673 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:41:19.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:41:19.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:41:19.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:41:20.587 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:41:20.587 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:41:21.477 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:41:21.478 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:41:21.478 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:41:21.478 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:41:21.478 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:41:22.032 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:41:22.466 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:41:22.588 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:41:24.971 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:41:24.971 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:41:24.971 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:41:24.971 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:41:24.971 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:41:25.658 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:41:25.681 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:41:25.692 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:41:25.723 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:41:25.752 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:41:25.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:41:25.813 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:41:25.842 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:41:25.869 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:41:25.895 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:41:25.924 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:41:25.950 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:41:25.951 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:41:25.951 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:41:25.951 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:41:25.951 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:41:25.952 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:41:25.952 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:41:25.952 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:41:25.952 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:41:25.952 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:41:25.952 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:41:25.952 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:41:25.953 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:41:25.953 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:41:26.560 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:41:26.561 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:41:27.936 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:41:28.491 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:41:28.561 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:41:30.768 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:41:31.279 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:41:31.324 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:41:32.141 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:41:32.183 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:41:32.217 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:41:32.249 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:41:32.280 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:41:32.312 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:41:32.338 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:41:32.367 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:41:32.393 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:41:32.419 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:41:32.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:41:32.421 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:41:32.421 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:41:32.421 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:41:32.421 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:41:32.421 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:41:32.421 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:41:32.422 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:41:32.422 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:41:32.422 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:41:32.422 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:41:32.422 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:41:32.422 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:41:33.135 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:41:33.135 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:41:33.564 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:41:33.565 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:41:33.565 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:41:33.565 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:41:33.565 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:41:34.132 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:41:35.136 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:41:35.491 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:41:36.708 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:41:36.709 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:41:36.709 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:41:36.709 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:41:37.394 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:41:37.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:41:37.711 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:41:37.734 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:41:37.757 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:41:37.782 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:41:37.808 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:41:37.834 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:41:37.860 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:41:37.887 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:41:37.933 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:41:37.966 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:41:37.968 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:41:37.968 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:41:37.968 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:41:37.968 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:41:37.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:41:37.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:41:37.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:41:37.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:41:37.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:41:37.969 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:41:37.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:41:37.970 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:41:37.970 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:41:38.211 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:41:38.540 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:41:38.540 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:41:40.043 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:41:40.541 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:41:40.572 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:41:41.358 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:41:43.144 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:41:43.171 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:41:43.195 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:41:43.219 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:41:43.246 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:41:43.272 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:41:43.300 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:41:43.325 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:41:43.354 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:41:43.381 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:41:43.406 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:41:43.408 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:41:43.408 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:41:43.408 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:41:43.408 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:41:43.409 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:41:43.409 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:41:43.409 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:41:43.409 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:41:43.409 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:41:43.409 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:41:43.409 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:41:43.410 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:41:43.410 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:41:43.434 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:41:43.434 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:41:43.434 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:41:43.435 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:41:43.435 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:41:44.062 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:41:44.062 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:41:44.211 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:41:44.409 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:41:46.063 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:41:46.441 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:41:46.973 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:41:47.209 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:41:47.209 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:41:48.958 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:41:48.980 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:41:49.008 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:41:49.034 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:41:49.061 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:41:49.087 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:41:49.113 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:41:49.139 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:41:49.169 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:41:49.207 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:41:49.234 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:41:49.235 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:41:49.236 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:41:49.236 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:41:49.236 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:41:49.236 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:41:49.236 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:41:49.236 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:41:49.237 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:41:49.237 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:41:49.237 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:41:49.237 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:41:49.237 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:41:49.237 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:41:49.732 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:41:49.861 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:41:49.861 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:41:50.070 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:41:50.749 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:41:50.749 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:41:51.862 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:41:52.380 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:41:52.687 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:41:53.630 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:41:53.630 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:41:54.373 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:41:54.401 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:41:54.427 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:41:54.463 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:41:54.490 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:41:54.531 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:41:54.558 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:41:54.584 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:41:54.629 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:41:54.672 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:41:54.707 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:41:54.708 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:41:54.708 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:41:54.708 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:41:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:41:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:41:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:41:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:41:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:41:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:41:54.709 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:41:54.710 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:41:54.710 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:41:54.710 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:41:55.140 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:41:55.409 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:41:55.410 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:41:55.935 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:41:56.625 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:41:56.625 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:41:57.410 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:41:57.739 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:41:59.359 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:41:59.974 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:41:59.974 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:42:00.324 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:42:00.346 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:42:00.369 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:42:00.393 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:42:00.419 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:42:00.445 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:42:00.473 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:42:00.500 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:42:00.528 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:42:00.555 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:42:00.587 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:42:00.587 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:42:00.587 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:42:00.587 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:42:00.587 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:42:00.588 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:42:00.588 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:42:00.588 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:42:00.588 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:42:00.588 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:42:00.588 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:42:00.588 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:42:00.589 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:42:00.589 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:42:00.603 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:42:01.338 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:42:01.339 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:42:02.284 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:42:03.018 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:42:03.018 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:42:03.340 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:42:04.373 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:42:05.710 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:42:06.229 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:42:06.255 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:42:06.271 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:42:06.272 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:42:06.279 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:42:06.304 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:42:06.332 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:42:06.356 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:42:06.383 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:42:06.430 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:42:06.478 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:42:06.517 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:42:06.559 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:42:06.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:42:06.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:42:06.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:42:06.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:42:06.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:42:06.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:42:06.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:42:06.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:42:06.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:42:06.562 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:42:06.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:42:06.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:42:06.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:42:07.064 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:42:07.172 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:42:07.172 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:42:08.940 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:42:09.173 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:42:09.564 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:42:09.564 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:42:10.373 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:42:11.924 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:42:11.947 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:42:11.973 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:42:11.997 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:42:12.023 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:42:12.049 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:42:12.075 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:42:12.100 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:42:12.126 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:42:12.153 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:42:12.166 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:42:12.179 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:42:12.180 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:42:12.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:42:12.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:42:12.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:42:12.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:42:12.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:42:12.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:42:12.182 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:42:12.182 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:42:12.182 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:42:12.182 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:42:12.182 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:42:12.182 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:42:12.799 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:42:12.800 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:42:12.937 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:42:12.937 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:42:13.325 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:42:14.939 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:42:15.123 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:42:15.955 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:42:15.956 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:42:16.405 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:42:18.210 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:42:18.241 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:42:18.266 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:42:18.290 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:42:18.316 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:42:18.343 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:42:18.372 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:42:18.401 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:42:18.427 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:42:18.451 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:42:18.452 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:42:18.479 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:42:18.479 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:42:18.480 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:42:18.480 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:42:18.480 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:42:18.480 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:42:18.480 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:42:18.480 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:42:18.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:42:18.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:42:18.481 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:42:18.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:42:18.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:42:18.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:42:19.066 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:42:19.066 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:42:19.222 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:42:19.222 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:42:19.291 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:42:21.223 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:42:21.320 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:42:22.114 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:42:22.114 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:42:22.140 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:42:24.059 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:42:24.091 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:42:24.116 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:42:24.149 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:42:24.180 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:42:24.210 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:42:24.239 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:42:24.266 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:42:24.293 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:42:24.320 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:42:24.358 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:42:24.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:42:24.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:42:24.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:42:24.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:42:24.362 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:42:24.362 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:42:24.362 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:42:24.362 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:42:24.362 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:42:24.362 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:42:24.362 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:42:24.362 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:42:24.363 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:42:24.714 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:42:24.860 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:42:24.940 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:42:24.940 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:42:25.349 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:42:25.350 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:42:26.942 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:42:27.582 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:42:28.165 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:42:28.166 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:42:28.244 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:42:29.922 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:42:29.922 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:42:29.922 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:42:29.922 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:42:29.937 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:42:29.964 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:42:29.991 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:42:30.017 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:42:30.049 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:42:30.076 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:42:30.106 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:42:30.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:42:30.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:42:30.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:42:30.108 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:42:30.108 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:42:30.108 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:42:30.108 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:42:30.108 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:42:30.108 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:42:30.108 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:42:30.109 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:42:30.109 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:42:30.109 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:42:30.655 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:42:30.656 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:42:30.903 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:42:31.018 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:42:31.501 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:42:31.501 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:42:32.656 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:42:34.389 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:42:34.403 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:42:35.339 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:42:35.340 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:42:35.604 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:42:35.626 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:42:35.650 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:42:35.674 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:42:35.701 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:42:35.727 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:42:35.753 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:42:35.779 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:42:35.807 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:42:35.835 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:42:35.862 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:42:35.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:42:35.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:42:35.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:42:35.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:42:35.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:42:35.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:42:35.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:42:35.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:42:35.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:42:35.866 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:42:35.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:42:35.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:42:35.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:42:36.394 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:42:36.395 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:42:37.007 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:42:37.370 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:42:38.162 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:42:38.163 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:42:38.395 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:42:40.393 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:42:40.931 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:42:40.952 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:42:40.993 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:42:41.025 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:42:41.060 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:42:41.129 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:42:41.153 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:42:41.165 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:42:41.193 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:42:41.221 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:42:41.248 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:42:41.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:42:41.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:42:41.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:42:41.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:42:41.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:42:41.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:42:41.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:42:41.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:42:41.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:42:41.251 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:42:41.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:42:41.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:42:41.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:42:41.566 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:42:41.795 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:42:41.795 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:42:42.125 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:42:42.125 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:42:43.214 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:42:43.796 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:42:44.027 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:42:45.308 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:42:45.308 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:42:46.338 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:42:46.361 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:42:46.385 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:42:46.408 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:42:46.437 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:42:46.463 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:42:46.489 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:42:46.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:42:46.541 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:42:46.568 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:42:46.620 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:42:46.623 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:42:46.623 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:42:46.624 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:42:46.624 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:42:46.624 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:42:46.624 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:42:46.624 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:42:46.625 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:42:46.625 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:42:46.625 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:42:46.625 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:42:46.625 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:42:46.625 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:42:46.920 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:42:47.372 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:42:47.372 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:42:47.954 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:42:48.491 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:42:48.492 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:42:49.374 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:42:49.756 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:42:50.659 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:42:51.682 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:42:51.683 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:42:52.133 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:42:52.157 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:42:52.179 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:42:52.208 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:42:52.241 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:42:52.271 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:42:52.300 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:42:52.330 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:42:52.357 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:42:52.390 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:42:52.417 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:42:52.419 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:42:52.419 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:42:52.419 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:42:52.419 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:42:52.419 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:42:52.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:42:52.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:42:52.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:42:52.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:42:52.420 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:42:52.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:42:52.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:42:52.421 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:42:53.164 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:42:53.165 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:42:54.290 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:42:54.291 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:42:54.291 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:42:54.291 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:42:54.431 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:42:54.954 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:42:55.166 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:42:57.096 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:42:57.533 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:42:57.533 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:42:57.533 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:42:57.534 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:42:57.534 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:42:57.674 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:42:58.321 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:42:58.537 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:42:58.569 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:42:58.604 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:42:58.639 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:42:58.670 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:42:58.705 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:42:58.731 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:42:58.758 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:42:58.784 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:42:58.811 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:42:58.812 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:42:58.812 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:42:58.812 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:42:58.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:42:58.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:42:58.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:42:58.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:42:58.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:42:58.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:42:58.813 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:42:58.814 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:42:58.814 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:42:58.814 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:42:59.719 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:42:59.719 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:43:00.407 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:43:01.087 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:43:01.569 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:43:01.721 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:43:02.977 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:43:03.888 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:43:04.431 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:43:04.664 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:43:04.686 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:43:04.709 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:43:04.733 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:43:04.759 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:43:04.787 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:43:04.818 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:43:04.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:43:04.874 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:43:04.902 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:43:04.928 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:43:04.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:43:04.932 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:43:04.932 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:43:04.933 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:43:04.933 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:43:04.933 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:43:04.933 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:43:04.933 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:43:04.934 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:43:04.934 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:43:04.934 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:43:04.934 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:43:04.934 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:43:05.747 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:43:05.747 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:43:06.946 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:43:06.946 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:43:06.946 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:43:06.946 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:43:06.946 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:43:07.421 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:43:07.472 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:43:07.748 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:43:09.920 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:43:10.077 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:43:10.077 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:43:10.077 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:43:10.077 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:43:10.077 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:43:10.161 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:43:10.184 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:43:10.206 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:43:10.230 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:43:10.257 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:43:10.283 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:43:10.309 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:43:10.335 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:43:10.362 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:43:10.400 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:43:10.434 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:43:10.436 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:43:10.436 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:43:10.436 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:43:10.436 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:43:10.437 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:43:10.437 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:43:10.437 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:43:10.438 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:43:10.438 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:43:10.439 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:43:10.439 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:43:10.439 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:43:10.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:43:10.801 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:43:11.121 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:43:11.121 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:43:12.990 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:43:13.122 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:43:13.404 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:43:13.579 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:43:16.018 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:43:16.041 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:43:16.069 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:43:16.095 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:43:16.121 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:43:16.148 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:43:16.163 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:43:16.178 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:43:16.207 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:43:16.235 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:43:16.262 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:43:16.303 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:43:16.303 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:43:16.303 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:43:16.304 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:43:16.304 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:43:16.308 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:43:16.310 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:43:16.310 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:43:16.310 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:43:16.310 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:43:16.310 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:43:16.311 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:43:16.311 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:43:16.311 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:43:16.311 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:43:16.311 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:43:16.311 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:43:16.311 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:43:16.312 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:43:16.968 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:43:17.231 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:43:17.232 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:43:19.233 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:43:19.370 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:43:19.370 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:43:19.371 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:43:19.371 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:43:19.371 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:43:19.860 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:43:20.095 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:43:22.096 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:43:22.128 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:43:22.171 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:43:22.215 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:43:22.250 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:43:22.281 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:43:22.308 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:43:22.336 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:43:22.363 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:43:22.391 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:43:22.417 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:43:22.419 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:43:22.419 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:43:22.419 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:43:22.419 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:43:22.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:43:22.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:43:22.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:43:22.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:43:22.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:43:22.420 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:43:22.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:43:22.421 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:43:22.421 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:43:22.572 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:43:22.628 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:43:22.996 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:43:22.997 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:43:23.485 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:43:24.998 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:43:25.425 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:43:26.673 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:43:27.329 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:43:27.592 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:43:27.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:43:27.638 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:43:27.661 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:43:27.687 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:43:27.713 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:43:27.738 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:43:27.765 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:43:27.793 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:43:27.820 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:43:27.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:43:27.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:43:27.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:43:27.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:43:27.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:43:27.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:43:27.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:43:27.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:43:27.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:43:27.851 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:43:27.851 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:43:27.851 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:43:27.851 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:43:27.851 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:43:28.240 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:43:28.391 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:43:28.391 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:43:29.790 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:43:29.790 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:43:29.790 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:43:29.790 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:43:29.790 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:43:30.393 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:43:30.476 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:43:31.425 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:43:33.179 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:43:33.753 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:43:33.754 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:43:34.369 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:43:36.088 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:43:36.749 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:43:36.750 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:43:37.352 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:43:38.865 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:43:39.415 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:43:39.415 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:43:39.993 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:43:41.812 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:43:42.426 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:43:42.427 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:43:42.922 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:43:42.951 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:43:42.977 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:43:43.000 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:43:43.027 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:43:43.054 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:43:43.080 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:43:43.117 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:43:43.145 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:43:43.156 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:43:43.187 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:43:43.216 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:43:43.217 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:43:43.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:43:43.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:43:43.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:43:43.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:43:43.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:43:43.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:43:43.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:43:43.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:43:43.219 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:43:43.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:43:43.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:43:43.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:43:44.411 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:43:44.412 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:43:44.944 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:43:45.494 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:43:45.495 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:43:45.946 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:43:46.413 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:43:48.396 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:43:49.187 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:43:49.188 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:43:49.431 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:43:49.533 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:43:49.556 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:43:49.579 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:43:49.603 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:43:49.631 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:43:49.658 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:43:49.684 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:43:49.711 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:43:49.738 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:43:49.765 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:43:49.791 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:43:49.794 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:43:49.794 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:43:49.794 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:43:49.794 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:43:49.794 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:43:49.794 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:43:49.794 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:43:49.795 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:43:49.795 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:43:49.795 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:43:49.795 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:43:49.795 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:43:49.795 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:43:50.550 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:43:50.551 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:43:51.547 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:43:52.082 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:43:52.089 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:43:52.089 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:43:52.552 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:43:55.214 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:43:55.517 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:43:55.780 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:43:55.807 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:43:55.816 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:43:55.816 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:43:55.832 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:43:55.857 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:43:55.893 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:43:55.923 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:43:55.950 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:43:55.976 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:43:56.002 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:43:56.030 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:43:56.055 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:43:56.056 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:43:56.057 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:43:56.057 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:43:56.057 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:43:56.057 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:43:56.057 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:43:56.057 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:43:56.057 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:43:56.058 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:43:56.058 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:43:56.058 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:43:56.058 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:43:56.058 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:43:56.771 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:43:56.772 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:43:58.217 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:43:58.256 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:43:58.773 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:43:58.787 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:43:58.788 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:44:01.661 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:44:01.687 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:44:01.712 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:44:01.744 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:44:01.770 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:44:01.797 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:44:01.823 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:44:01.849 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:44:01.875 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:44:01.902 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:44:01.921 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:44:01.928 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:44:01.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:44:01.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:44:01.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:44:01.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:44:01.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:44:01.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:44:01.931 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:44:01.931 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:44:01.931 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:44:01.931 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:44:01.931 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:44:01.931 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:44:01.931 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:44:02.611 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:44:02.611 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:44:02.767 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:44:02.767 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:44:04.650 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:44:04.768 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:44:05.190 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:44:05.191 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:44:07.264 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:44:07.300 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:44:07.328 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:44:07.353 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:44:07.380 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:44:07.406 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:44:07.432 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:44:07.458 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:44:07.467 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:44:07.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:44:07.512 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:44:07.539 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:44:07.541 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:44:07.541 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:44:07.541 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:44:07.541 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:44:07.542 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:44:07.542 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:44:07.542 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:44:07.542 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:44:07.542 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:44:07.542 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:44:07.543 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:44:07.543 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:44:07.543 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:44:08.154 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:44:08.154 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:44:08.309 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:44:08.309 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:44:10.160 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:44:10.311 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:44:10.775 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:44:10.776 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:44:11.944 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:44:13.145 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:44:13.169 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:44:13.193 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:44:13.217 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:44:13.227 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:44:13.256 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:44:13.270 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:44:13.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:44:13.323 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:44:13.349 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:44:13.376 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:44:13.402 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:44:13.412 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:44:13.412 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:44:13.412 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:44:13.413 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:44:13.413 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:44:13.413 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:44:13.413 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:44:13.413 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:44:13.413 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:44:13.413 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:44:13.414 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:44:13.414 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:44:13.414 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:44:13.783 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:44:13.784 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:44:14.150 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:44:14.151 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:44:14.849 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:44:16.151 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:44:16.218 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:44:17.006 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:44:17.007 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:44:17.964 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:44:19.351 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:44:19.377 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:44:19.401 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:44:19.425 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:44:19.452 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:44:19.478 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:44:19.504 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:44:19.520 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:44:19.530 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:44:19.556 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:44:19.583 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:44:19.609 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:44:19.611 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:44:19.611 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:44:19.611 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:44:19.611 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:44:19.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:44:19.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:44:19.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:44:19.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:44:19.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:44:19.612 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:44:19.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:44:19.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:44:19.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:44:20.260 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:44:20.260 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:44:20.431 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:44:20.431 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:44:20.874 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:44:22.432 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:44:22.696 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:44:23.450 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:44:23.450 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:44:24.478 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:44:25.675 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:44:25.699 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:44:25.722 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:44:25.746 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:44:25.773 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:44:25.799 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:44:25.825 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:44:25.857 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:44:25.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:44:25.923 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:44:25.954 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:44:25.955 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:44:25.955 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:44:25.955 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:44:25.955 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:44:25.956 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:44:25.956 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:44:25.956 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:44:25.956 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:44:25.956 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:44:25.956 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:44:25.956 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:44:25.957 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:44:25.957 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:44:26.172 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:44:26.774 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:44:26.775 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:44:26.956 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:44:26.956 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:44:27.424 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:44:28.776 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:44:29.356 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:44:30.103 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:44:30.103 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:44:30.464 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:44:31.892 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:44:31.926 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:44:31.953 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:44:31.977 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:44:32.003 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:44:32.030 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:44:32.058 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:44:32.086 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:44:32.114 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:44:32.163 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:44:32.204 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:44:32.207 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:44:32.207 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:44:32.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:44:32.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:44:32.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:44:32.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:44:32.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:44:32.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:44:32.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:44:32.209 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:44:32.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:44:32.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:44:32.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:44:32.777 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:44:32.778 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:44:32.815 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:44:33.234 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:44:33.303 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:44:33.304 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:44:34.779 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:44:36.056 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:44:36.492 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:44:36.866 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:44:36.866 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:44:37.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:44:37.804 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:44:37.828 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:44:37.851 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:44:37.878 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:44:37.904 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:44:37.930 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:44:37.962 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:44:37.991 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:44:38.019 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:44:38.046 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:44:38.047 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:44:38.047 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:44:38.048 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:44:38.048 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:44:38.048 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:44:38.048 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:44:38.048 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:44:38.048 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:44:38.048 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:44:38.049 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:44:38.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:44:38.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:44:38.049 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:44:38.803 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:44:38.803 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:44:39.418 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:44:39.418 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:44:39.418 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:44:39.419 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:44:39.445 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:44:39.965 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:44:40.804 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:44:42.854 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:44:42.854 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:44:42.855 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:44:42.855 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:44:42.855 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:44:43.156 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:44:43.316 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:44:43.340 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:44:43.344 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:44:43.367 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:44:43.391 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:44:43.419 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:44:43.445 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:44:43.472 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:44:43.498 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:44:43.524 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:44:43.553 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:44:43.580 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:44:43.581 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:44:43.581 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:44:43.582 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:44:43.582 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:44:43.582 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:44:43.583 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:44:43.583 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:44:43.583 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:44:43.583 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:44:43.583 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:44:43.583 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:44:43.583 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:44:43.583 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:44:44.221 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:44:44.221 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:44:45.715 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:44:45.732 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:44:46.222 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:44:46.300 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:44:49.248 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:44:49.461 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:44:49.467 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:44:49.490 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:44:49.513 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:44:49.536 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:44:49.574 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:44:49.610 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:44:49.639 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:44:49.666 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:44:49.693 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:44:49.720 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:44:49.746 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:44:49.748 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:44:49.748 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:44:49.748 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:44:49.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:44:49.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:44:49.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:44:49.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:44:49.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:44:49.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:44:49.750 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:44:49.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:44:49.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:44:49.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:44:49.781 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:44:50.613 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:44:50.614 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:44:52.109 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:44:52.109 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:44:52.109 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:44:52.109 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:44:52.109 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:44:52.319 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:44:52.616 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:44:52.664 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:44:55.642 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:44:55.665 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:44:55.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:44:55.713 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:44:55.740 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:44:55.747 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:44:55.766 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:44:55.792 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:44:55.820 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:44:55.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:44:55.873 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:44:55.900 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:44:55.902 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:44:55.902 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:44:55.902 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:44:55.902 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:44:55.902 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:44:55.902 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:44:55.903 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:44:55.903 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:44:55.903 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:44:55.903 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:44:55.903 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:44:55.903 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:44:55.904 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:44:56.453 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:44:56.596 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:44:56.596 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:44:58.543 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:44:58.598 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:44:59.125 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:45:01.295 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:45:01.319 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:45:01.349 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:45:01.373 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:45:01.399 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:45:01.425 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:45:01.453 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:45:01.480 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:45:01.507 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:45:01.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:45:01.560 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:45:01.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:45:01.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:45:01.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:45:01.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:45:01.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:45:01.563 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:45:01.563 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:45:01.563 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:45:01.563 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:45:01.563 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:45:01.563 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:45:01.563 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:45:01.564 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:45:01.641 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:45:01.641 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:45:01.642 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:45:01.642 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:45:01.642 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:45:02.219 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:45:02.219 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:45:02.393 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:45:04.220 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:45:04.500 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:45:04.501 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:45:04.501 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:45:04.501 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:45:04.501 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:45:05.186 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:45:05.482 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:45:07.409 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:45:07.434 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:45:07.459 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:45:07.484 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:45:07.509 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:45:07.535 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:45:07.563 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:45:07.589 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:45:07.619 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:45:07.646 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:45:07.672 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:45:07.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:45:07.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:45:07.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:45:07.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:45:07.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:45:07.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:45:07.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:45:07.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:45:07.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:45:07.675 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:45:07.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:45:07.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:45:07.676 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:45:07.742 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:45:08.299 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:45:08.300 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:45:08.481 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:45:08.501 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:45:10.300 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:45:11.061 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:45:11.487 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:45:11.692 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:45:13.326 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:45:14.078 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:45:14.176 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:45:14.223 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:45:14.261 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:45:14.292 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:45:14.320 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:45:14.347 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:45:14.374 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:45:14.401 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:45:14.428 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:45:14.450 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:45:14.457 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:45:14.457 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:45:14.457 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:45:14.457 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:45:14.457 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:45:14.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:45:14.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:45:14.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:45:14.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:45:14.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:45:14.458 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:45:14.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:45:14.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:45:14.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:45:14.636 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:45:14.970 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:45:14.971 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:45:16.972 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:45:17.076 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:45:17.511 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:45:17.886 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:45:17.886 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:45:19.864 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:45:20.381 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:45:20.440 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:45:20.716 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:45:20.755 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:45:20.784 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:45:20.814 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:45:20.842 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:45:20.869 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:45:20.898 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:45:20.926 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:45:20.953 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:45:20.978 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:45:20.979 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:45:20.988 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:45:20.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:45:20.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:45:20.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:45:20.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:45:20.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:45:20.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:45:20.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:45:20.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:45:20.991 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:45:20.991 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:45:20.991 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:45:20.991 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:45:20.991 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:45:21.548 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:45:21.549 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:45:23.439 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:45:23.477 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:45:23.550 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:45:24.174 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:45:24.174 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:45:26.385 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:45:26.405 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:45:26.408 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:45:26.433 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:45:26.455 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:45:26.481 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:45:26.504 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:45:26.521 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:45:26.553 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:45:26.585 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:45:26.618 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:45:26.645 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:45:26.671 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:45:26.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:45:26.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:45:26.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:45:26.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:45:26.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:45:26.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:45:26.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:45:26.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:45:26.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:45:26.674 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:45:26.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:45:26.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:45:26.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:45:26.951 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:45:26.951 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:45:27.518 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:45:27.518 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:45:29.491 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:45:29.519 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:45:29.525 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:45:30.432 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:45:30.432 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:45:32.173 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:45:32.563 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:45:33.242 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:45:33.242 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:45:35.264 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:45:35.529 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:45:35.782 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:45:35.782 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:45:37.897 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:45:38.202 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:45:38.481 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:45:38.481 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:45:40.783 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:45:40.960 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:45:41.507 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:45:41.508 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:45:42.547 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:45:43.410 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:45:43.456 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:45:43.491 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:45:43.528 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:45:43.561 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:45:43.574 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:45:43.593 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:45:43.623 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:45:43.654 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:45:43.682 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:45:43.686 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:45:43.710 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:45:43.712 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:45:43.712 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:45:43.712 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:45:43.712 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:45:43.712 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:45:43.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:45:43.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:45:43.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:45:43.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:45:43.713 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:45:43.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:45:43.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:45:43.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:45:44.146 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:45:44.147 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:45:44.736 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:45:44.737 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:45:46.728 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:45:46.738 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:45:46.952 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:45:47.377 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:45:47.378 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:45:49.449 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:45:49.712 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:45:49.769 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:45:50.291 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:45:50.292 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:45:50.300 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:45:50.345 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:45:50.378 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:45:50.409 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:45:50.438 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:45:50.467 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:45:50.497 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:45:50.525 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:45:50.572 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:45:50.608 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:45:50.609 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:45:50.609 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:45:50.609 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:45:50.609 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:45:50.610 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:45:50.610 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:45:50.610 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:45:50.610 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:45:50.610 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:45:50.610 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:45:50.610 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:45:50.611 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:45:50.611 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:45:51.491 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:45:51.492 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:45:52.751 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:45:53.272 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:45:53.273 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:45:53.493 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:45:53.522 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:45:56.049 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:45:56.320 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:45:56.555 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:45:56.583 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:45:56.584 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:45:57.421 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:45:57.467 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:45:57.508 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:45:57.541 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:45:57.569 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:45:57.596 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:45:57.625 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:45:57.652 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:45:57.692 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:45:57.740 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:45:57.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:45:57.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:45:57.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:45:57.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:45:57.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:45:57.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:45:57.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:45:57.743 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:45:57.743 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:45:57.743 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:45:57.743 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:45:57.743 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:45:57.743 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:45:58.522 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:45:58.523 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:45:59.221 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:45:59.550 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:45:59.790 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:45:59.791 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:46:00.523 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:46:02.183 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:46:02.214 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:46:02.939 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:46:02.939 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:46:03.190 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:46:03.214 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:46:03.239 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:46:03.263 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:46:03.289 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:46:03.315 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:46:03.341 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:46:03.370 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:46:03.396 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:46:03.423 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:46:03.449 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:46:03.452 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:46:03.452 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:46:03.452 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:46:03.452 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:46:03.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:46:03.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:46:03.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:46:03.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:46:03.453 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:46:03.453 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:46:03.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:46:03.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:46:03.454 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:46:04.346 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:46:04.346 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:46:05.544 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:46:05.562 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:46:06.250 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:46:06.250 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:46:06.347 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:46:08.425 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:46:08.502 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:46:09.055 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:46:09.062 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:46:09.063 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:46:09.081 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:46:09.115 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:46:09.139 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:46:09.166 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:46:09.192 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:46:09.226 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:46:09.244 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:46:09.271 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:46:09.298 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:46:09.325 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:46:09.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:46:09.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:46:09.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:46:09.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:46:09.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:46:09.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:46:09.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:46:09.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:46:09.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:46:09.329 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:46:09.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:46:09.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:46:09.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:46:10.247 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:46:10.248 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:46:11.703 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:46:12.215 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:46:12.216 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:46:12.249 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:46:12.520 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:46:14.816 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:46:14.839 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:46:14.863 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:46:14.887 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:46:14.914 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:46:14.941 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:46:14.967 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:46:14.994 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:46:15.020 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:46:15.047 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:46:15.073 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:46:15.074 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:46:15.075 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:46:15.075 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:46:15.075 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:46:15.075 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:46:15.075 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:46:15.075 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:46:15.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:46:15.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:46:15.076 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:46:15.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:46:15.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:46:15.076 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:46:15.130 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:46:15.409 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:46:15.731 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:46:15.732 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:46:15.916 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:46:15.916 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:46:17.733 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:46:18.125 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:46:18.555 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:46:18.793 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:46:18.793 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:46:20.254 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:46:20.279 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:46:20.307 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:46:20.337 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:46:20.359 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:46:20.386 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:46:20.412 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:46:20.439 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:46:20.464 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:46:20.491 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:46:20.517 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:46:20.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:46:20.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:46:20.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:46:20.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:46:20.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:46:20.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:46:20.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:46:20.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:46:20.520 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:46:20.520 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:46:20.520 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:46:20.520 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:46:20.520 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:46:20.975 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:46:20.975 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:46:21.435 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:46:21.598 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:46:22.174 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:46:22.976 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:46:24.835 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:46:25.491 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:46:25.542 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:46:26.125 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:46:26.978 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:46:27.028 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:46:27.069 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:46:27.106 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:46:27.161 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:46:27.208 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:46:27.249 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:46:27.287 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:46:27.322 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:46:27.353 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:46:27.355 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:46:27.356 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:46:27.356 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:46:27.356 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:46:27.356 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:46:27.356 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:46:27.356 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:46:27.356 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:46:27.357 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:46:27.357 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:46:27.357 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:46:27.357 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:46:27.357 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:46:27.835 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:46:27.835 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:46:27.881 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:46:27.882 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:46:27.882 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:46:27.882 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:46:27.882 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:46:28.087 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:46:28.333 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:46:29.836 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:46:30.971 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:46:30.972 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:46:30.972 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:46:30.972 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:46:31.550 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:46:31.767 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:46:32.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:46:32.571 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:46:32.609 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:46:32.636 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:46:32.665 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:46:32.692 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:46:32.719 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:46:32.745 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:46:32.788 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:46:32.815 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:46:32.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:46:32.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:46:32.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:46:32.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:46:32.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:46:32.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:46:32.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:46:32.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:46:32.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:46:32.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:46:32.850 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:46:32.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:46:32.851 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:46:32.851 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:46:33.574 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:46:33.574 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:46:34.142 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:46:34.221 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:46:34.732 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:46:35.576 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:46:37.313 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:46:37.584 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:46:37.978 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:46:38.622 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:46:38.660 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:46:38.693 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:46:38.716 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:46:38.742 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:46:38.773 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:46:38.799 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:46:38.826 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:46:38.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:46:38.879 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:46:38.907 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:46:38.908 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:46:38.909 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:46:38.909 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:46:38.909 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:46:38.909 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:46:38.909 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:46:38.909 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:46:38.909 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:46:38.910 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:46:38.910 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:46:38.910 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:46:38.910 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:46:38.910 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:46:39.610 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:46:39.611 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:46:40.205 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:46:40.205 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:46:40.205 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:46:40.205 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:46:40.357 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:46:40.778 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:46:41.612 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:46:43.341 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:46:43.342 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:46:43.342 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:46:43.342 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:46:43.342 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:46:43.618 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:46:44.222 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:46:44.879 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:46:44.916 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:46:44.951 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:46:44.982 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:46:45.012 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:46:45.038 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:46:45.064 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:46:45.094 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:46:45.121 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:46:45.148 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:46:45.179 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:46:45.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:46:45.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:46:45.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:46:45.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:46:45.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:46:45.181 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:46:45.182 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:46:45.182 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:46:45.182 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:46:45.182 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:46:45.182 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:46:45.182 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:46:45.182 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:46:45.906 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:46:45.907 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:46:46.475 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:46:46.476 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:46:47.028 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:46:47.907 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:46:49.542 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:46:49.584 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:46:50.662 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:46:51.214 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:46:52.090 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:46:52.125 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:46:52.156 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:46:52.201 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:46:52.238 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:46:52.271 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:46:52.303 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:46:52.331 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:46:52.377 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:46:52.417 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:46:52.418 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:46:52.418 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:46:52.419 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:46:52.419 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:46:52.419 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:46:52.419 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:46:52.419 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:46:52.419 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:46:52.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:46:52.420 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:46:52.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:46:52.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:46:52.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:46:52.583 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:46:52.584 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:46:52.584 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:46:52.584 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:46:52.584 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:46:52.663 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:46:52.905 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:46:52.905 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:46:53.288 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:46:54.907 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:46:55.471 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:46:55.472 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:46:55.472 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:46:55.472 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:46:55.472 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:46:55.568 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:46:56.051 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:46:57.682 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:46:57.719 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:46:57.751 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:46:57.775 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:46:57.801 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:46:57.828 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:46:57.860 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:46:57.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:46:57.918 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:46:57.946 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:46:57.972 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:46:57.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:46:57.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:46:57.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:46:57.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:46:57.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:46:57.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:46:57.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:46:57.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:46:57.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:46:57.975 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:46:57.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:46:57.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:46:57.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:46:58.434 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:46:58.499 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:46:58.499 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:46:58.670 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:46:58.869 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:46:58.869 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:47:00.500 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:47:01.175 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:47:01.199 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:47:01.931 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:47:01.931 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:47:03.254 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:47:03.277 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:47:03.304 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:47:03.331 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:47:03.359 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:47:03.385 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:47:03.413 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:47:03.439 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:47:03.466 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:47:03.493 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:47:03.519 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:47:03.522 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:47:03.522 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:47:03.522 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:47:03.522 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:47:03.522 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:47:03.522 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:47:03.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:47:03.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:47:03.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:47:03.523 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:47:03.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:47:03.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:47:03.523 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:47:04.027 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:47:04.028 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:47:04.103 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:47:04.480 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:47:05.184 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:47:05.185 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:47:06.028 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:47:06.987 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:47:07.783 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:47:08.464 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:47:08.465 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:47:08.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:47:08.938 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:47:08.967 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:47:08.996 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:47:09.024 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:47:09.050 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:47:09.077 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:47:09.103 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:47:09.130 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:47:09.157 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:47:09.184 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:47:09.185 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:47:09.186 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:47:09.186 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:47:09.186 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:47:09.186 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:47:09.186 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:47:09.186 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:47:09.187 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:47:09.187 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:47:09.187 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:47:09.187 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:47:09.187 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:47:09.187 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:47:09.770 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:47:09.771 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:47:09.793 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:47:11.062 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:47:11.743 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:47:11.743 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:47:11.772 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:47:12.560 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:47:14.338 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:47:14.371 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:47:14.397 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:47:14.421 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:47:14.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:47:14.474 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:47:14.499 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:47:14.522 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:47:14.529 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:47:14.556 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:47:14.584 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:47:14.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:47:14.616 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:47:14.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:47:14.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:47:14.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:47:14.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:47:14.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:47:14.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:47:14.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:47:14.618 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:47:14.618 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:47:14.618 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:47:14.618 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:47:14.618 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:47:15.234 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:47:15.234 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:47:15.405 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:47:15.405 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:47:15.560 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:47:17.407 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:47:17.541 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:47:18.149 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:47:18.150 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:47:18.807 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:47:20.533 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:47:20.778 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:47:21.329 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:47:21.329 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:47:21.390 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:47:21.440 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:47:21.480 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:47:21.521 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:47:21.546 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:47:21.574 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:47:21.601 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:47:21.628 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:47:21.654 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:47:21.655 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:47:21.682 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:47:21.683 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:47:21.683 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:47:21.683 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:47:21.683 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:47:21.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:47:21.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:47:21.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:47:21.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:47:21.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:47:21.684 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:47:21.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:47:21.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:47:21.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:47:22.664 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:47:22.665 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:47:23.814 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:47:24.356 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:47:24.356 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:47:24.666 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:47:24.828 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:47:26.797 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:47:27.441 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:47:27.442 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:47:27.473 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:47:27.815 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:47:27.844 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:47:27.871 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:47:27.897 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:47:27.924 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:47:27.952 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:47:28.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:47:28.045 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:47:28.081 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:47:28.109 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:47:28.138 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:47:28.140 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:47:28.140 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:47:28.140 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:47:28.141 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:47:28.141 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:47:28.141 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:47:28.141 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:47:28.141 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:47:28.141 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:47:28.141 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:47:28.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:47:28.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:47:28.142 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:47:28.968 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:47:28.968 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:47:29.984 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:47:30.576 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:47:30.586 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:47:30.586 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:47:30.969 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:47:33.067 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:47:33.160 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:47:33.182 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:47:33.187 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:47:33.214 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:47:33.238 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:47:33.276 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:47:33.310 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:47:33.361 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:47:33.405 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:47:33.435 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:47:33.462 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:47:33.488 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:47:33.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:47:33.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:47:33.491 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:47:33.491 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:47:33.491 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:47:33.491 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:47:33.491 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:47:33.491 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:47:33.492 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:47:33.492 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:47:33.492 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:47:33.492 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:47:33.492 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:47:33.591 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:47:33.592 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:47:34.160 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:47:34.160 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:47:36.161 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:47:36.203 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:47:36.609 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:47:36.807 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:47:36.807 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:47:39.100 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:47:39.292 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:47:39.405 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:47:39.967 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:47:39.973 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:47:39.973 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:47:40.010 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:47:40.047 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:47:40.075 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:47:40.102 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:47:40.129 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:47:40.155 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:47:40.182 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:47:40.209 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:47:40.236 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:47:40.237 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:47:40.237 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:47:40.237 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:47:40.238 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:47:40.238 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:47:40.238 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:47:40.238 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:47:40.238 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:47:40.238 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:47:40.238 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:47:40.239 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:47:40.239 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:47:40.239 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:47:40.936 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:47:40.936 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:47:42.549 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:47:42.623 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:47:42.938 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:47:43.259 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:47:43.260 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:47:45.406 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:47:45.429 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:47:45.442 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:47:45.452 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:47:45.477 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:47:45.504 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:47:45.530 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:47:45.560 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:47:45.586 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:47:45.613 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:47:45.644 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:47:45.682 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:47:45.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:47:45.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:47:45.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:47:45.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:47:45.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:47:45.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:47:45.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:47:45.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:47:45.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:47:45.685 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:47:45.686 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:47:45.686 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:47:45.686 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:47:45.691 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:47:46.632 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:47:46.632 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:47:46.633 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:47:46.634 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:47:48.635 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:47:48.781 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:47:48.976 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:47:49.334 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:47:49.334 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:47:51.592 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:47:51.829 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:47:51.919 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:47:52.370 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:47:52.370 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:47:52.453 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:47:52.477 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:47:52.501 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:47:52.529 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:47:52.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:47:52.586 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:47:52.630 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:47:52.656 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:47:52.683 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:47:52.709 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:47:52.710 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:47:52.710 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:47:52.711 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:47:52.711 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:47:52.711 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:47:52.711 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:47:52.711 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:47:52.711 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:47:52.711 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:47:52.712 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:47:52.712 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:47:52.712 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:47:52.713 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:47:53.929 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:47:53.929 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:47:55.007 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:47:55.551 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:47:55.552 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:47:55.588 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:47:55.930 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:47:58.087 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:47:58.152 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:47:58.625 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:47:58.625 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:47:58.701 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:47:58.724 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:47:58.747 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:47:58.773 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:47:58.800 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:47:58.827 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:47:58.854 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:47:58.885 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:47:58.918 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:47:58.948 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:47:58.975 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:47:58.977 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:47:58.977 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:47:58.977 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:47:58.977 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:47:58.978 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:47:58.978 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:47:58.978 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:47:58.978 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:47:58.978 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:47:58.978 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:47:58.978 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:47:58.979 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:47:58.979 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:47:59.768 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:47:59.768 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:48:01.180 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:48:01.583 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:48:01.728 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:48:01.728 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:48:01.769 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:48:04.105 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:48:04.308 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:48:04.345 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:48:04.368 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:48:04.372 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:48:04.372 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:48:04.372 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:48:04.372 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:48:04.390 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:48:04.417 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:48:04.442 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:48:04.521 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:48:04.562 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:48:04.613 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:48:04.653 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:48:04.691 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:48:04.691 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:48:04.691 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:48:04.691 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:48:04.691 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:48:04.692 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:48:04.692 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:48:04.692 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:48:04.692 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:48:04.692 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:48:04.692 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:48:04.692 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:48:04.693 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:48:04.693 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:48:05.132 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:48:05.771 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:48:05.771 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:48:06.989 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:48:07.773 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:48:07.891 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:48:07.892 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:48:07.892 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:48:07.892 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:48:07.892 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:48:08.427 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:48:09.593 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:48:10.538 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:48:10.836 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:48:11.391 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:48:11.440 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:48:11.481 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:48:11.522 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:48:11.536 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:48:11.552 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:48:11.580 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:48:11.607 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:48:11.634 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:48:11.662 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:48:11.688 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:48:11.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:48:11.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:48:11.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:48:11.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:48:11.690 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:48:11.691 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:48:11.691 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:48:11.691 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:48:11.691 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:48:11.691 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:48:11.691 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:48:11.692 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:48:11.692 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:48:12.353 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:48:12.353 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:48:12.595 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:48:14.199 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:48:14.355 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:48:14.727 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:48:15.352 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:48:17.432 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:48:17.511 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:48:17.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:48:17.558 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:48:17.599 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:48:17.609 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:48:17.636 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:48:17.662 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:48:17.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:48:17.715 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:48:17.763 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:48:17.802 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:48:17.804 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:48:17.804 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:48:17.804 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:48:17.805 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:48:17.805 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:48:17.805 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:48:17.805 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:48:17.805 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:48:17.805 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:48:17.805 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:48:17.806 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:48:17.806 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:48:17.806 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:48:18.038 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:48:18.445 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:48:18.445 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:48:18.578 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:48:20.279 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:48:20.280 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:48:20.280 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:48:20.280 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:48:20.280 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:48:20.446 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:48:20.884 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:48:21.250 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:48:23.143 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:48:23.181 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:48:23.218 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:48:23.252 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:48:23.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:48:23.319 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:48:23.349 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:48:23.376 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:48:23.403 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:48:23.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:48:23.486 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:48:23.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:48:23.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:48:23.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:48:23.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:48:23.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:48:23.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:48:23.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:48:23.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:48:23.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:48:23.489 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:48:23.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:48:23.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:48:23.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:48:23.654 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:48:24.134 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:48:24.134 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:48:24.577 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:48:25.615 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:48:26.135 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:48:26.950 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:48:27.728 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:48:28.297 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:48:28.313 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:48:29.155 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:48:29.228 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:48:29.260 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:48:29.291 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:48:29.328 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:48:29.344 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:48:29.371 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:48:29.402 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:48:29.430 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:48:29.456 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:48:29.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:48:29.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:48:29.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:48:29.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:48:29.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:48:29.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:48:29.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:48:29.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:48:29.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:48:29.459 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:48:29.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:48:29.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:48:29.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:48:30.137 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:48:30.137 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:48:30.197 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:48:30.197 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:48:30.197 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:48:30.197 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:48:30.197 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:48:30.695 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:48:32.000 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:48:32.138 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:48:32.935 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:48:32.935 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:48:32.935 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:48:32.935 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:48:33.509 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:48:34.763 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:48:34.821 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:48:35.501 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:48:35.501 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:48:35.501 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:48:35.501 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:48:35.502 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:48:35.678 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:48:35.717 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:48:35.744 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:48:35.770 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:48:35.799 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:48:35.826 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:48:35.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:48:35.883 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:48:35.907 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:48:35.934 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:48:35.935 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:48:35.936 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:48:35.936 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:48:35.936 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:48:35.936 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:48:35.936 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:48:35.937 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:48:35.937 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:48:35.937 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:48:35.937 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:48:35.937 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:48:35.937 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:48:35.938 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:48:36.127 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:48:36.870 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:48:36.870 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:48:38.026 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:48:38.760 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:48:38.871 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:48:39.260 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:48:40.695 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:48:41.219 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:48:41.244 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:48:41.266 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:48:41.289 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:48:41.320 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:48:41.346 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:48:41.374 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:48:41.400 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:48:41.413 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:48:41.427 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:48:41.454 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:48:41.480 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:48:41.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:48:41.483 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:48:41.483 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:48:41.483 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:48:41.483 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:48:41.483 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:48:41.483 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:48:41.484 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:48:41.484 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:48:41.484 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:48:41.484 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:48:41.484 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:48:41.484 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:48:42.080 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:48:42.080 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:48:42.526 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:48:42.526 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:48:44.319 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:48:44.527 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:48:44.605 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:48:44.908 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:48:44.908 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:48:47.225 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:48:47.247 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:48:47.270 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:48:47.295 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:48:47.321 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:48:47.343 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:48:47.347 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:48:47.374 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:48:47.399 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:48:47.415 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:48:47.426 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:48:47.458 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:48:47.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:48:47.486 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:48:47.486 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:48:47.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:48:47.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:48:47.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:48:47.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:48:47.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:48:47.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:48:47.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:48:47.488 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:48:47.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:48:47.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:48:47.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:48:47.982 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:48:47.983 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:48:48.290 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:48:48.290 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:48:50.272 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:48:50.291 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:48:50.618 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:48:50.949 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:48:50.949 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:48:53.110 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:48:53.134 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:48:53.158 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:48:53.194 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:48:53.220 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:48:53.247 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:48:53.273 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:48:53.300 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:48:53.327 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:48:53.333 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:48:53.354 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:48:53.368 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:48:53.383 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:48:53.384 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:48:53.384 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:48:53.384 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:48:53.384 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:48:53.384 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:48:53.385 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:48:53.385 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:48:53.385 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:48:53.385 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:48:53.385 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:48:53.385 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:48:53.385 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:48:53.386 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:48:54.026 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:48:54.026 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:48:54.305 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:48:54.305 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:48:56.306 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:48:56.453 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:48:56.698 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:48:57.134 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:48:57.134 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:48:59.170 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:48:59.203 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:48:59.233 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:48:59.277 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:48:59.315 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:48:59.346 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:48:59.379 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:48:59.406 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:48:59.433 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:48:59.460 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:48:59.486 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:48:59.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:48:59.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:48:59.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:48:59.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:48:59.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:48:59.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:48:59.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:48:59.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:48:59.489 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:48:59.490 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:48:59.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:48:59.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:48:59.490 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:48:59.533 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:48:59.579 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:49:00.188 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:49:00.189 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:49:00.447 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:49:00.447 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:49:02.191 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:49:02.616 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:49:02.848 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:49:03.806 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:49:03.806 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:49:04.740 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:49:05.514 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:49:05.594 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:49:05.637 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:49:05.672 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:49:05.704 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:49:05.740 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:49:05.771 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:49:05.804 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:49:05.842 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:49:05.881 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:49:05.895 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:49:05.913 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:49:05.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:49:05.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:49:05.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:49:05.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:49:05.915 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:49:05.916 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:49:05.916 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:49:05.916 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:49:05.916 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:49:05.916 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:49:05.916 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:49:05.917 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:49:05.917 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:49:06.807 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:49:06.807 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:49:07.062 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:49:07.062 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:49:08.925 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:49:09.003 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:49:09.063 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:49:09.623 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:49:09.624 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:49:11.622 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:49:11.848 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:49:11.929 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:49:12.436 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:49:12.436 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:49:12.491 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:49:12.526 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:49:12.555 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:49:12.592 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:49:12.625 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:49:12.655 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:49:12.684 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:49:12.711 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:49:12.738 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:49:12.763 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:49:12.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:49:12.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:49:12.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:49:12.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:49:12.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:49:12.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:49:12.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:49:12.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:49:12.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:49:12.766 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:49:12.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:49:12.767 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:49:12.767 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:49:13.773 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:49:13.773 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:49:14.863 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:49:15.395 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:49:15.396 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:49:15.630 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:49:15.774 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:49:17.998 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:49:18.439 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:49:18.616 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:49:18.616 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:49:18.724 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:49:18.762 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:49:18.785 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:49:18.808 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:49:18.834 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:49:18.860 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:49:18.886 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:49:18.912 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:49:18.938 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:49:18.966 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:49:18.991 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:49:18.993 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:49:18.993 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:49:18.993 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:49:18.993 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:49:18.993 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:49:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:49:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:49:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:49:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:49:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:49:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:49:18.994 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:49:18.995 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:49:19.733 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:49:19.733 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:49:21.391 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:49:21.658 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:49:21.734 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:49:21.922 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:49:21.923 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:49:24.322 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:49:24.409 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:49:24.617 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:49:25.264 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:49:25.311 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:49:25.312 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:49:25.318 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:49:25.358 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:49:25.396 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:49:25.424 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:49:25.451 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:49:25.477 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:49:25.504 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:49:25.531 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:49:25.558 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:49:25.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:49:25.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:49:25.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:49:25.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:49:25.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:49:25.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:49:25.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:49:25.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:49:25.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:49:25.561 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:49:25.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:49:25.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:49:25.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:49:26.257 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:49:26.257 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:49:27.063 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:49:28.074 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:49:28.258 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:49:28.734 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:49:28.734 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:49:29.549 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:49:31.102 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:49:31.130 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:49:31.147 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:49:31.170 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:49:31.196 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:49:31.222 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:49:31.257 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:49:31.388 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:49:31.388 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:49:31.389 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:49:31.414 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:49:31.418 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:49:31.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:49:31.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:49:31.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:49:31.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:49:31.420 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:49:31.421 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:49:31.421 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:49:31.421 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:49:31.421 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:49:31.421 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:49:31.421 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:49:31.421 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:49:31.422 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:49:32.053 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:49:32.054 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:49:32.210 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:49:32.210 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:49:33.629 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:49:34.211 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:49:34.275 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:49:34.856 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:49:34.856 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:49:36.237 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:49:36.799 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:49:37.056 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:49:37.642 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:49:37.643 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:49:37.658 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:49:37.699 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:49:37.735 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:49:37.772 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:49:37.822 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:49:37.871 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:49:37.903 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:49:37.929 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:49:37.956 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:49:37.989 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:49:37.991 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:49:37.991 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:49:37.991 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:49:37.991 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:49:37.992 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:49:37.992 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:49:37.992 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:49:37.992 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:49:37.992 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:49:37.992 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:49:37.992 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:49:37.993 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:49:37.993 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:49:38.750 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:49:38.750 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:49:39.897 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:49:40.047 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:49:40.733 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:49:40.734 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:49:40.751 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:49:42.566 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:49:42.982 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:49:42.983 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:49:42.999 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:49:43.022 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:49:43.048 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:49:43.077 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:49:43.103 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:49:43.144 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:49:43.145 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:49:43.177 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:49:43.209 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:49:43.240 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:49:43.241 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:49:43.242 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:49:43.242 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:49:43.242 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:49:43.242 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:49:43.242 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:49:43.242 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:49:43.242 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:49:43.243 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:49:43.243 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:49:43.243 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:49:43.243 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:49:43.243 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:49:44.048 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:49:44.049 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:49:44.125 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:49:44.125 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:49:45.921 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:49:46.126 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:49:46.160 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:49:46.160 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:49:46.160 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:49:46.160 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:49:47.112 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:49:48.508 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:49:48.887 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:49:48.911 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:49:48.934 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:49:48.957 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:49:48.983 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:49:49.009 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:49:49.036 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:49:49.063 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:49:49.090 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:49:49.117 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:49:49.144 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:49:49.146 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:49:49.146 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:49:49.146 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:49:49.147 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:49:49.147 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:49:49.147 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:49:49.147 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:49:49.147 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:49:49.147 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:49:49.147 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:49:49.148 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:49:49.148 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:49:49.148 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:49:49.746 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:49:49.823 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:49:49.823 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:49:50.477 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:49:51.648 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:49:51.825 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:49:52.851 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:49:53.594 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:49:54.196 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:49:54.205 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:49:55.070 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:49:55.103 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:49:55.131 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:49:55.158 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:49:55.212 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:49:55.252 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:49:55.281 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:49:55.311 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:49:55.338 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:49:55.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:49:55.366 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:49:55.367 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:49:55.367 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:49:55.367 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:49:55.367 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:49:55.367 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:49:55.367 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:49:55.367 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:49:55.368 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:49:55.368 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:49:55.368 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:49:55.368 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:49:55.368 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:49:55.839 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:49:55.840 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:49:55.840 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:49:55.840 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:49:56.387 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:49:56.387 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:49:56.671 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:49:57.701 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:49:58.388 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:49:58.797 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:49:58.798 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:49:58.798 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:49:58.798 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:49:58.798 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:49:59.549 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:50:00.446 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:50:00.688 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:50:00.712 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:50:00.735 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:50:00.758 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:50:00.785 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:50:00.811 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:50:00.837 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:50:00.863 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:50:00.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:50:00.918 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:50:00.944 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:50:00.946 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:50:00.947 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:50:00.947 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:50:00.947 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:50:00.947 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:50:00.947 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:50:00.947 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:50:00.948 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:50:00.948 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:50:00.948 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:50:00.948 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:50:00.948 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:50:00.948 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:50:01.707 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:50:01.707 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:50:02.434 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:50:02.963 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:50:03.709 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:50:04.020 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:50:05.699 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:50:06.346 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:50:06.378 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:50:06.383 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:50:06.404 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:50:06.428 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:50:06.455 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:50:06.488 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:50:06.527 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:50:06.561 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:50:06.592 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:50:06.621 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:50:06.646 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:50:06.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:50:06.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:50:06.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:50:06.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:50:06.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:50:06.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:50:06.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:50:06.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:50:06.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:50:06.649 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:50:06.650 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:50:06.650 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:50:06.650 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:50:06.903 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:50:07.236 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:50:07.236 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:50:09.042 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:50:09.042 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:50:09.042 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:50:09.042 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:50:09.043 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:50:09.237 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:50:09.669 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:50:09.674 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:50:12.060 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:50:12.089 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:50:12.116 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:50:12.141 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:50:12.167 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:50:12.194 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:50:12.220 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:50:12.246 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:50:12.273 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:50:12.281 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:50:12.300 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:50:12.327 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:50:12.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:50:12.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:50:12.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:50:12.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:50:12.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:50:12.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:50:12.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:50:12.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:50:12.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:50:12.330 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:50:12.330 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:50:12.330 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:50:12.330 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:50:12.422 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:50:12.988 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:50:13.254 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:50:13.254 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:50:15.254 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:50:15.305 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:50:15.664 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:50:15.847 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:50:18.426 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:50:18.450 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:50:19.119 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:50:21.663 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:50:21.677 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:50:22.327 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:50:24.448 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:50:24.491 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:50:25.048 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:50:25.048 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:50:27.217 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:50:27.377 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:50:27.750 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:50:27.750 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:50:28.051 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:50:28.079 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:50:28.102 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:50:28.125 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:50:28.151 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:50:28.177 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:50:28.202 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:50:28.228 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:50:28.254 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:50:28.281 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:50:28.307 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:50:28.316 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:50:28.316 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:50:28.316 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:50:28.317 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:50:28.317 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:50:28.317 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:50:28.317 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:50:28.317 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:50:28.317 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:50:28.317 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:50:28.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:50:28.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:50:28.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:50:28.831 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:50:28.831 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:50:29.793 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:50:30.082 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:50:30.338 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:50:30.339 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:50:30.832 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:50:33.194 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:50:33.818 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:50:33.819 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:50:33.869 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:50:33.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:50:33.915 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:50:33.937 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:50:33.963 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:50:33.990 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:50:34.015 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:50:34.043 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:50:34.069 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:50:34.096 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:50:34.122 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:50:34.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:50:34.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:50:34.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:50:34.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:50:34.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:50:34.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:50:34.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:50:34.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:50:34.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:50:34.126 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:50:34.126 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:50:34.126 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:50:34.126 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:50:34.812 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:50:34.812 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:50:35.843 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:50:36.398 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:50:36.398 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:50:36.812 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:50:38.850 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:50:39.423 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:50:39.423 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:50:39.456 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:50:39.492 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:50:39.522 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:50:39.555 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:50:39.581 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:50:39.607 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:50:39.632 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:50:39.659 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:50:39.685 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:50:39.713 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:50:39.738 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:50:39.740 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:50:39.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:50:39.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:50:39.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:50:39.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:50:39.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:50:39.741 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:50:39.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:50:39.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:50:39.742 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:50:39.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:50:39.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:50:39.742 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:50:40.431 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:50:40.431 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:50:41.527 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:50:42.090 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:50:42.090 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:50:42.432 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:50:43.732 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:50:44.854 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:50:45.369 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:50:45.397 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:50:45.403 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:50:45.403 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:50:45.422 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:50:45.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:50:45.473 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:50:45.499 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:50:45.526 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:50:45.552 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:50:45.579 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:50:45.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:50:45.632 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:50:45.633 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:50:45.633 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:50:45.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:50:45.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:50:45.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:50:45.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:50:45.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:50:45.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:50:45.635 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:50:45.635 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:50:45.635 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:50:45.635 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:50:45.635 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:50:46.276 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:50:46.276 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:50:46.494 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:50:47.733 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:50:48.277 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:50:48.412 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:50:48.412 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:50:49.679 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:50:51.139 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:50:51.416 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:50:51.938 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:50:51.938 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:50:51.995 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:50:52.041 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:50:52.088 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:50:52.126 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:50:52.174 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:50:52.216 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:50:52.248 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:50:52.278 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:50:52.318 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:50:52.354 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:50:52.356 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:50:52.356 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:50:52.356 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:50:52.356 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:50:52.357 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:50:52.357 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:50:52.357 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:50:52.357 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:50:52.357 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:50:52.357 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:50:52.358 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:50:52.358 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:50:52.358 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:50:52.696 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:50:52.989 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:50:52.989 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:50:54.478 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:50:54.990 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:50:55.155 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:50:55.156 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:50:55.716 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:50:57.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:50:57.676 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:50:57.707 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:50:57.732 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:50:57.759 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:50:57.785 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:50:57.811 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:50:57.838 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:50:57.865 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:50:57.888 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:50:57.902 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:50:57.938 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:50:57.939 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:50:57.940 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:50:57.940 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:50:57.940 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:50:57.940 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:50:57.940 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:50:57.940 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:50:57.940 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:50:57.941 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:50:57.941 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:50:57.941 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:50:57.941 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:50:57.941 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:50:58.494 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:50:58.494 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:50:58.657 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:50:58.657 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:50:58.768 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:51:00.659 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:51:00.794 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:51:01.346 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:51:01.347 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:51:02.154 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:51:03.710 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:51:03.737 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:51:04.264 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:51:04.265 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:51:04.563 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:51:04.587 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:51:04.612 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:51:04.638 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:51:04.664 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:51:04.690 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:51:04.716 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:51:04.742 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:51:04.771 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:51:04.797 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:51:04.799 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:51:04.799 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:51:04.799 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:51:04.799 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:51:04.799 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:51:04.799 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:51:04.800 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:51:04.800 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:51:04.800 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:51:04.800 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:51:04.800 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:51:04.800 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:51:04.800 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:51:05.154 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:51:05.342 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:51:05.342 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:51:06.549 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:51:07.318 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:51:07.318 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:51:07.343 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:51:07.905 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:51:09.952 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:51:09.975 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:51:10.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:51:10.026 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:51:10.052 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:51:10.080 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:51:10.108 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:51:10.133 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:51:10.161 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:51:10.189 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:51:10.215 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:51:10.217 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:51:10.217 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:51:10.217 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:51:10.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:51:10.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:51:10.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:51:10.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:51:10.218 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:51:10.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:51:10.219 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:51:10.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:51:10.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:51:10.219 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:51:10.233 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:51:10.617 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:51:10.841 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:51:10.841 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:51:11.027 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:51:11.027 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:51:12.842 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:51:13.215 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:51:13.237 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:51:14.164 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:51:14.164 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:51:15.384 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:51:15.406 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:51:15.429 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:51:15.452 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:51:15.478 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:51:15.505 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:51:15.537 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:51:15.585 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:51:15.627 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:51:15.663 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:51:15.694 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:51:15.701 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:51:15.701 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:51:15.701 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:51:15.701 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:51:15.701 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:51:15.701 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:51:15.702 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:51:15.702 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:51:15.702 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:51:15.702 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:51:15.702 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:51:15.702 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:51:15.702 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:51:15.995 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:51:16.323 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:51:16.323 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:51:16.553 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:51:17.123 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:51:17.123 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:51:18.324 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:51:18.685 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:51:19.142 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:51:19.883 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:51:19.883 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:51:21.195 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:51:21.222 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:51:21.250 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:51:21.273 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:51:21.300 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:51:21.326 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:51:21.352 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:51:21.378 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:51:21.404 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:51:21.442 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:51:21.444 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:51:21.479 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:51:21.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:51:21.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:51:21.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:51:21.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:51:21.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:51:21.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:51:21.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:51:21.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:51:21.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:51:21.483 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:51:21.483 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:51:21.483 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:51:21.483 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:51:22.047 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:51:22.047 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:51:22.709 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:51:23.479 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:51:23.479 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:51:24.048 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:51:25.051 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:51:26.050 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:51:26.740 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:51:26.741 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:51:27.121 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:51:27.973 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:51:28.021 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:51:28.062 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:51:28.116 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:51:28.162 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:51:28.200 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:51:28.233 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:51:28.234 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:51:28.272 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:51:28.302 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:51:28.330 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:51:28.331 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:51:28.332 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:51:28.332 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:51:28.332 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:51:28.332 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:51:28.332 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:51:28.332 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:51:28.332 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:51:28.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:51:28.333 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:51:28.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:51:28.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:51:28.333 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:51:28.874 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:51:28.874 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:51:28.874 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:51:28.875 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:51:29.030 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:51:29.030 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:51:29.465 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:51:31.031 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:51:31.702 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:51:31.801 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:51:31.802 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:51:31.802 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:51:31.802 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:51:31.802 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:51:32.582 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:51:33.765 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:51:33.788 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:51:33.811 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:51:33.833 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:51:33.860 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:51:33.886 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:51:33.913 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:51:33.940 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:51:33.966 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:51:33.993 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:51:34.020 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:51:34.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:51:34.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:51:34.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:51:34.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:51:34.022 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:51:34.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:51:34.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:51:34.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:51:34.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:51:34.023 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:51:34.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:51:34.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:51:34.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:51:34.419 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:51:34.576 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:51:34.576 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:51:35.185 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:51:35.784 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:51:36.577 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:51:37.741 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:51:38.497 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:51:39.055 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:51:39.595 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:51:39.631 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:51:39.664 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:51:39.695 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:51:39.721 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:51:39.765 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:51:39.802 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:51:39.836 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:51:39.863 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:51:39.889 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:51:39.918 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:51:39.919 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:51:39.920 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:51:39.920 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:51:39.920 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:51:39.920 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:51:39.920 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:51:39.920 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:51:39.920 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:51:39.921 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:51:39.921 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:51:39.921 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:51:39.921 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:51:39.921 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:51:40.486 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:51:40.612 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:51:40.613 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:51:41.699 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:51:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:51:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:51:41.700 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:51:41.700 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:51:42.409 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:51:42.614 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:51:43.756 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:51:45.048 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:51:45.048 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:51:45.048 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:51:45.048 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:51:45.048 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:51:45.184 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:51:45.207 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:51:45.230 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:51:45.253 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:51:45.279 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:51:45.306 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:51:45.332 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:51:45.358 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:51:45.385 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:51:45.411 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:51:45.437 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:51:45.439 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:51:45.439 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:51:45.439 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:51:45.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:51:45.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:51:45.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:51:45.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:51:45.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:51:45.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:51:45.440 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:51:45.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:51:45.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:51:45.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:51:45.553 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:51:45.911 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:51:45.911 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:51:46.581 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:51:47.912 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:51:47.931 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:51:48.572 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:51:49.714 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:51:50.977 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:51:51.003 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:51:51.031 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:51:51.059 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:51:51.086 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:51:51.112 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:51:51.138 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:51:51.164 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:51:51.190 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:51:51.217 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:51:51.226 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:51:51.226 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:51:51.226 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:51:51.226 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:51:51.226 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:51:51.243 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:51:51.245 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:51:51.245 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:51:51.245 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:51:51.246 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:51:51.246 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:51:51.246 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:51:51.246 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:51:51.246 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:51:51.246 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:51:51.246 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:51:51.246 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:51:51.247 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:51:51.247 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:51:51.881 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:51:52.030 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:51:52.030 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:51:52.440 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:51:54.032 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:51:54.232 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:51:54.232 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:51:54.232 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:51:54.232 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:51:54.233 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:51:55.005 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:51:55.742 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:51:57.114 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:51:57.144 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:51:57.168 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:51:57.192 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:51:57.218 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:51:57.244 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:51:57.270 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:51:57.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:51:57.322 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:51:57.350 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:51:57.376 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:51:57.378 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:51:57.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:51:57.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:51:57.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:51:57.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:51:57.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:51:57.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:51:57.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:51:57.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:51:57.380 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:51:57.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:51:57.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:51:57.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:51:57.921 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:51:57.922 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:51:57.925 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:51:58.328 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:51:58.374 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:51:59.923 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:52:01.100 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:52:01.736 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:52:01.937 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:52:02.790 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:52:02.815 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:52:02.840 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:52:02.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:52:02.896 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:52:02.922 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:52:02.948 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:52:02.974 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:52:03.000 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:52:03.031 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:52:03.058 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:52:03.059 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:52:03.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:52:03.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:52:03.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:52:03.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:52:03.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:52:03.060 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:52:03.061 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:52:03.061 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:52:03.061 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:52:03.061 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:52:03.061 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:52:03.061 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:52:03.785 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:52:03.786 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:52:04.368 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:52:04.368 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:52:04.368 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:52:04.368 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:52:04.369 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:52:04.519 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:52:04.954 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:52:05.787 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:52:07.380 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:52:07.547 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:52:08.376 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:52:08.376 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:52:08.762 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:52:08.784 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:52:08.807 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:52:08.830 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:52:08.857 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:52:08.884 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:52:08.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:52:08.936 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:52:08.963 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:52:08.990 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:52:09.021 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:52:09.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:52:09.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:52:09.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:52:09.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:52:09.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:52:09.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:52:09.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:52:09.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:52:09.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:52:09.024 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:52:09.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:52:09.025 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:52:09.025 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:52:09.926 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:52:09.927 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:52:10.093 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:52:10.434 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:52:11.151 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:52:11.151 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:52:11.928 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:52:12.731 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:52:13.912 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:52:14.416 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:52:14.416 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:52:14.991 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:52:15.035 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:52:15.073 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:52:15.098 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:52:15.124 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:52:15.153 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:52:15.180 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:52:15.207 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:52:15.246 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:52:15.266 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:52:15.293 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:52:15.295 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:52:15.295 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:52:15.295 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:52:15.295 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:52:15.295 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:52:15.295 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:52:15.295 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:52:15.296 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:52:15.296 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:52:15.296 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:52:15.296 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:52:15.296 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:52:15.296 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:52:15.547 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:52:16.157 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:52:16.158 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:52:16.947 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:52:17.525 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:52:17.525 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:52:18.158 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:52:18.727 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:52:20.283 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:52:20.792 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:52:20.793 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:52:20.797 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:52:20.831 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:52:20.860 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:52:20.883 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:52:20.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:52:20.935 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:52:20.961 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:52:20.987 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:52:21.015 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:52:21.042 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:52:21.068 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:52:21.070 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:52:21.070 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:52:21.070 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:52:21.070 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:52:21.071 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:52:21.071 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:52:21.071 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:52:21.071 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:52:21.071 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:52:21.071 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:52:21.071 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:52:21.072 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:52:21.072 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:52:21.461 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:52:21.797 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:52:21.798 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:52:23.168 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:52:23.799 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:52:23.861 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:52:23.861 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:52:24.738 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:52:26.286 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:52:26.537 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:52:26.563 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:52:26.587 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:52:26.612 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:52:26.638 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:52:26.663 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:52:26.690 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:52:26.717 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:52:26.743 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:52:26.783 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:52:26.818 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:52:26.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:52:26.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:52:26.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:52:26.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:52:26.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:52:26.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:52:26.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:52:26.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:52:26.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:52:26.821 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:52:26.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:52:26.822 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:52:26.822 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:52:26.886 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:52:26.887 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:52:27.283 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:52:27.283 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:52:27.606 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:52:29.190 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:52:29.284 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:52:29.882 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:52:29.882 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:52:30.741 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:52:32.185 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:52:32.209 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:52:32.232 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:52:32.256 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:52:32.282 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:52:32.309 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:52:32.341 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:52:32.372 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:52:32.401 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:52:32.429 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:52:32.444 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:52:32.456 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:52:32.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:52:32.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:52:32.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:52:32.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:52:32.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:52:32.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:52:32.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:52:32.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:52:32.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:52:32.459 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:52:32.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:52:32.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:52:32.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:52:33.090 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:52:33.090 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:52:33.236 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:52:33.236 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:52:33.546 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:52:35.237 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:52:35.373 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:52:35.797 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:52:35.798 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:52:36.772 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:52:38.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:52:38.310 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:52:38.334 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:52:38.359 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:52:38.386 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:52:38.423 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:52:38.453 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:52:38.483 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:52:38.516 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:52:38.547 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:52:38.574 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:52:38.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:52:38.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:52:38.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:52:38.576 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:52:38.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:52:38.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:52:38.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:52:38.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:52:38.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:52:38.577 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:52:38.577 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:52:38.578 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:52:38.578 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:52:38.605 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:52:39.240 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:52:39.240 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:52:39.326 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:52:39.414 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:52:39.414 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:52:41.241 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:52:42.004 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:52:42.152 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:52:42.933 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:52:42.933 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:52:43.629 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:52:43.659 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:52:43.683 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:52:43.708 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:52:43.736 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:52:43.763 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:52:43.790 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:52:43.816 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:52:43.859 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:52:43.887 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:52:43.914 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:52:43.916 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:52:43.916 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:52:43.916 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:52:43.917 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:52:43.917 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:52:43.917 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:52:43.917 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:52:43.917 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:52:43.917 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:52:43.917 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:52:43.918 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:52:43.918 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:52:43.918 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:52:44.694 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:52:44.694 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:52:45.172 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:52:45.283 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:52:45.815 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:52:45.815 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:52:46.696 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:52:48.666 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:52:49.041 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:52:49.285 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:52:49.285 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:52:49.730 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:52:49.757 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:52:49.779 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:52:49.803 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:52:49.830 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:52:49.856 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:52:49.882 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:52:49.911 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:52:49.937 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:52:49.967 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:52:49.995 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:52:49.996 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:52:49.996 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:52:49.997 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:52:49.997 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:52:49.997 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:52:49.997 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:52:49.997 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:52:49.997 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:52:49.997 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:52:49.998 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:52:49.998 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:52:49.998 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:52:49.998 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:52:50.573 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:52:50.573 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:52:51.477 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:52:51.747 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:52:51.971 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:52:51.971 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:52:52.575 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:52:54.697 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:52:55.175 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:52:55.310 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:52:55.310 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:52:55.885 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:52:55.910 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:52:55.935 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:52:55.959 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:52:55.986 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:52:56.013 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:52:56.040 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:52:56.066 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:52:56.093 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:52:56.127 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:52:56.154 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:52:56.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:52:56.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:52:56.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:52:56.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:52:56.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:52:56.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:52:56.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:52:56.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:52:56.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:52:56.157 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:52:56.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:52:56.158 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:52:56.158 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:52:56.897 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:52:56.897 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:52:57.729 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:52:57.798 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:52:58.240 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:52:58.240 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:52:58.899 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:53:00.759 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:53:00.960 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:53:01.760 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:53:01.760 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:53:02.135 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:53:02.157 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:53:02.180 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:53:02.202 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:53:02.229 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:53:02.255 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:53:02.281 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:53:02.307 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:53:02.334 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:53:02.360 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:53:02.386 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:53:02.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:53:02.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:53:02.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:53:02.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:53:02.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:53:02.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:53:02.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:53:02.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:53:02.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:53:02.389 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:53:02.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:53:02.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:53:02.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:53:03.235 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:53:03.235 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:53:03.489 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:53:03.934 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:53:04.653 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:53:04.653 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:53:05.236 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:53:07.043 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:53:07.547 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:53:08.089 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:53:08.113 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:53:08.136 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:53:08.162 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:53:08.162 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:53:08.166 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:53:08.191 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:53:08.218 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:53:08.244 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:53:08.270 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:53:08.297 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:53:08.323 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:53:08.349 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:53:08.351 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:53:08.351 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:53:08.351 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:53:08.351 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:53:08.351 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:53:08.351 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:53:08.351 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:53:08.352 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:53:08.352 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:53:08.352 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:53:08.352 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:53:08.352 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:53:08.352 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:53:08.982 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:53:08.982 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:53:09.789 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:53:10.159 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:53:10.905 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:53:10.905 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:53:10.983 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:53:13.165 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:53:13.923 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:53:13.995 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:53:14.019 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:53:14.042 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:53:14.071 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:53:14.096 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:53:14.122 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:53:14.149 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:53:14.175 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:53:14.202 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:53:14.229 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:53:14.256 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:53:14.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:53:14.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:53:14.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:53:14.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:53:14.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:53:14.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:53:14.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:53:14.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:53:14.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:53:14.260 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:53:14.260 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:53:14.260 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:53:14.260 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:53:14.437 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:53:14.849 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:53:14.849 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:53:15.802 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:53:16.434 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:53:16.435 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:53:16.435 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:53:16.435 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:53:16.435 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:53:16.850 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:53:16.969 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:53:19.588 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:53:19.615 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:53:19.642 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:53:19.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:53:19.693 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:53:19.719 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:53:19.745 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:53:19.760 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:53:19.766 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:53:19.781 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:53:19.807 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:53:19.835 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:53:19.861 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:53:19.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:53:19.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:53:19.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:53:19.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:53:19.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:53:19.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:53:19.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:53:19.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:53:19.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:53:19.864 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:53:19.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:53:19.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:53:19.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:53:20.866 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:53:20.870 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:53:20.870 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:53:22.459 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:53:22.847 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:53:22.871 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:53:23.661 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:53:25.793 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:53:26.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:53:26.518 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:53:26.547 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:53:26.571 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:53:26.589 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:53:26.590 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:53:26.590 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:53:26.590 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:53:26.590 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:53:26.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:53:26.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:53:26.673 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:53:26.703 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:53:26.747 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:53:26.787 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:53:26.820 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:53:26.822 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:53:26.823 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:53:26.823 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:53:26.823 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:53:26.823 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:53:26.823 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:53:26.823 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:53:26.823 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:53:26.824 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:53:26.824 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:53:26.824 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:53:26.824 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:53:26.824 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:53:27.073 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:53:27.453 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:53:27.454 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:53:28.533 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:53:28.947 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:53:28.947 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:53:28.947 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:53:28.947 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:53:28.947 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:53:29.455 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:53:29.680 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:53:31.780 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:53:32.357 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:53:32.392 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:53:32.422 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:53:32.452 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:53:32.481 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:53:32.512 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:53:32.539 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:53:32.567 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:53:32.599 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:53:32.629 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:53:32.685 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:53:32.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:53:32.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:53:32.686 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:53:32.686 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:53:32.686 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:53:32.686 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:53:32.686 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:53:32.686 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:53:32.687 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:53:32.687 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:53:32.687 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:53:32.687 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:53:32.688 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:53:32.893 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:53:33.244 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:53:33.245 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:53:33.561 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:53:34.495 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:53:35.246 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:53:35.984 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:53:36.858 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:53:38.051 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:53:38.621 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:53:38.647 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:53:38.675 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:53:38.699 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:53:38.726 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:53:38.752 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:53:38.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:53:38.820 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:53:38.859 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:53:38.891 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:53:38.916 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:53:38.918 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:53:38.919 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:53:38.919 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:53:38.919 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:53:38.919 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:53:38.919 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:53:38.919 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:53:38.919 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:53:38.920 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:53:38.920 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:53:38.920 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:53:38.920 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:53:38.920 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:53:39.584 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:53:39.584 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:53:39.584 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:53:39.585 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:53:39.585 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:53:39.648 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:53:39.648 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:53:40.069 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:53:40.851 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:53:41.649 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:53:42.374 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:53:42.374 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:53:42.374 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:53:42.374 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:53:43.050 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:53:44.718 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:53:44.791 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:53:45.571 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:53:45.612 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:53:45.646 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:53:45.680 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:53:45.711 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:53:45.711 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:53:45.738 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:53:45.765 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:53:45.791 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:53:45.823 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:53:45.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:53:45.854 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:53:45.855 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:53:45.855 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:53:45.855 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:53:45.855 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:53:45.855 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:53:45.855 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:53:45.856 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:53:45.856 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:53:45.856 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:53:45.856 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:53:45.856 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:53:45.856 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:53:46.413 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:53:46.568 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:53:46.568 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:53:47.452 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:53:48.554 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:53:48.569 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:53:49.511 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:53:50.784 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:53:51.602 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:53:51.642 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:53:51.673 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:53:51.701 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:53:51.728 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:53:51.757 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:53:51.784 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:53:51.814 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:53:51.841 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:53:51.869 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:53:51.895 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:53:51.897 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:53:51.897 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:53:51.898 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:53:51.898 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:53:51.898 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:53:51.898 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:53:51.898 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:53:51.898 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:53:51.898 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:53:51.899 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:53:51.899 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:53:51.899 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:53:51.899 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:53:52.487 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:53:52.559 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:53:52.559 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:53:52.901 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:53:52.902 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:53:53.821 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:53:54.561 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:53:55.064 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:53:55.983 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:53:55.983 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:53:57.062 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:53:57.583 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:53:57.583 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:53:57.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:53:57.638 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:53:57.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:53:57.739 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:53:57.775 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:53:57.808 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:53:57.841 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:53:57.873 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:53:57.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:53:57.907 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:53:57.907 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:53:57.907 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:53:57.908 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:53:57.908 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:53:57.908 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:53:57.908 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:53:57.908 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:53:57.909 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:53:57.909 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:53:57.909 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:53:57.909 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:53:57.909 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:53:58.457 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:53:58.458 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:53:58.702 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:53:59.146 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:53:59.147 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:53:59.907 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:54:00.459 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:54:01.810 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:54:02.567 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:54:02.567 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:54:03.149 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:54:03.171 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:54:03.194 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:54:03.217 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:54:03.254 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:54:03.289 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:54:03.321 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:54:03.351 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:54:03.379 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:54:03.406 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:54:03.440 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:54:03.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:54:03.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:54:03.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:54:03.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:54:03.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:54:03.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:54:03.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:54:03.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:54:03.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:54:03.442 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:54:03.442 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:54:03.443 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:54:03.443 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:54:03.800 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:54:04.217 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:54:04.217 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:54:05.190 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:54:05.713 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:54:05.714 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:54:06.219 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:54:06.628 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:54:08.483 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:54:09.197 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:54:09.198 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:54:09.427 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:54:09.797 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:54:10.278 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:54:10.325 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:54:10.367 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:54:10.397 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:54:10.433 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:54:10.465 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:54:10.495 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:54:10.523 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:54:10.550 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:54:10.577 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:54:10.580 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:54:10.580 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:54:10.580 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:54:10.580 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:54:10.581 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:54:10.581 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:54:10.581 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:54:10.581 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:54:10.581 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:54:10.581 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:54:10.581 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:54:10.582 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:54:10.582 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:54:11.088 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:54:11.089 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:54:11.626 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:54:12.136 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:54:12.136 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:54:12.382 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:54:13.090 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:54:14.720 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:54:15.561 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:54:15.561 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:54:15.810 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:54:17.853 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:54:18.373 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:54:18.374 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:54:18.521 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:54:20.603 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:54:21.159 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:54:21.159 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:54:21.832 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:54:23.365 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:54:23.882 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:54:23.882 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:54:24.559 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:54:26.131 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:54:26.137 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:54:26.162 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:54:26.185 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:54:26.224 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:54:26.253 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:54:26.279 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:54:26.306 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:54:26.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:54:26.359 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:54:26.387 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:54:26.413 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:54:26.416 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:54:26.416 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:54:26.416 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:54:26.416 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:54:26.417 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:54:26.417 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:54:26.417 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:54:26.417 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:54:26.418 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:54:26.418 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:54:26.418 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:54:26.419 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:54:26.419 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:54:26.669 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:54:26.670 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:54:27.206 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:54:27.206 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:54:27.910 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:54:29.094 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:54:29.207 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:54:29.625 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:54:29.625 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:54:30.721 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:54:31.696 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:54:31.719 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:54:31.743 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:54:31.766 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:54:31.793 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:54:31.820 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:54:31.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:54:31.877 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:54:31.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:54:31.932 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:54:31.968 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:54:31.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:54:31.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:54:31.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:54:31.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:54:31.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:54:31.969 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:54:31.970 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:54:31.970 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:54:31.970 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:54:31.970 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:54:31.970 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:54:31.970 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:54:31.970 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:54:32.301 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:54:32.938 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:54:32.938 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:54:33.087 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:54:33.087 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:54:33.807 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:54:34.939 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:54:35.245 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:54:35.952 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:54:35.952 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:54:36.489 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:54:37.182 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:54:37.205 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:54:37.228 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:54:37.251 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:54:37.278 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:54:37.318 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:54:37.334 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:54:37.360 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:54:37.386 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:54:37.412 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:54:37.437 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:54:37.439 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:54:37.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:54:37.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:54:37.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:54:37.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:54:37.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:54:37.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:54:37.440 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:54:37.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:54:37.441 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:54:37.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:54:37.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:54:37.441 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:54:38.306 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:54:38.307 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:54:38.920 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:54:39.491 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:54:39.491 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:54:39.814 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:54:40.307 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:54:42.036 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:54:42.530 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:54:42.618 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:54:42.619 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:54:42.872 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:54:43.723 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:54:43.751 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:54:43.779 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:54:43.809 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:54:43.834 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:54:43.877 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:54:43.915 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:54:43.948 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:54:43.978 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:54:44.004 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:54:44.006 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:54:44.006 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:54:44.007 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:54:44.007 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:54:44.007 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:54:44.007 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:54:44.007 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:54:44.007 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:54:44.008 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:54:44.008 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:54:44.008 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:54:44.008 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:54:44.008 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:54:44.771 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:54:44.771 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:54:45.339 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:54:45.848 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:54:45.857 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:54:45.857 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:54:46.772 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:54:48.199 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:54:48.332 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:54:48.823 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:54:48.824 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:54:49.510 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:54:50.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:54:50.426 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:54:50.464 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:54:50.501 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:54:50.535 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:54:50.563 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:54:50.597 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:54:50.624 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:54:50.652 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:54:50.679 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:54:50.681 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:54:50.681 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:54:50.681 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:54:50.681 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:54:50.682 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:54:50.682 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:54:50.682 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:54:50.682 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:54:50.682 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:54:50.682 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:54:50.683 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:54:50.683 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:54:50.683 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:54:51.326 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:54:51.326 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:54:51.595 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:54:51.826 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:54:52.115 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:54:52.115 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:54:53.326 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:54:54.105 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:54:54.491 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:54:54.659 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:54:54.659 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:54:56.802 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:54:56.838 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:54:56.864 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:54:56.897 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:54:56.924 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:54:56.951 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:54:56.983 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:54:57.015 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:54:57.037 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:54:57.065 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:54:57.089 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:54:57.090 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:54:57.090 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:54:57.090 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:54:57.090 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:54:57.090 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:54:57.090 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:54:57.091 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:54:57.091 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:54:57.091 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:54:57.091 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:54:57.091 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:54:57.091 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:54:57.091 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:54:57.654 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:54:57.677 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:54:57.677 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:54:58.069 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:54:58.153 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:54:59.678 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:55:00.123 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:55:00.566 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:55:00.849 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:55:02.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:55:02.488 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:55:02.513 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:55:02.537 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:55:02.564 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:55:02.593 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:55:02.619 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:55:02.645 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:55:02.673 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:55:02.719 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:55:02.759 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:55:02.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:55:02.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:55:02.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:55:02.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:55:02.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:55:02.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:55:02.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:55:02.763 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:55:02.763 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:55:02.763 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:55:02.763 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:55:02.763 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:55:02.763 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:55:03.603 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:55:03.603 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:55:03.910 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:55:03.910 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:55:03.910 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:55:03.910 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:55:03.911 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:55:04.429 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:55:04.832 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:55:05.604 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:55:06.723 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:55:06.723 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:55:06.723 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:55:06.723 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:55:07.492 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:55:07.535 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:55:07.866 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:55:07.889 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:55:07.912 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:55:07.936 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:55:07.962 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:55:07.987 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:55:08.014 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:55:08.039 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:55:08.065 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:55:08.092 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:55:08.118 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:55:08.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:55:08.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:55:08.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:55:08.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:55:08.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:55:08.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:55:08.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:55:08.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:55:08.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:55:08.122 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:55:08.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:55:08.123 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:55:08.123 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:55:08.867 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:55:08.867 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:55:10.098 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:55:10.741 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:55:10.834 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:55:10.868 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:55:13.302 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:55:13.317 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:55:13.326 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:55:13.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:55:13.400 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:55:13.437 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:55:13.465 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:55:13.492 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:55:13.518 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:55:13.545 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:55:13.572 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:55:13.596 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:55:13.596 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:55:13.596 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:55:13.596 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:55:13.596 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:55:13.598 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:55:13.600 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:55:13.600 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:55:13.600 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:55:13.600 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:55:13.601 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:55:13.601 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:55:13.601 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:55:13.601 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:55:13.601 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:55:13.601 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:55:13.601 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:55:13.602 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:55:13.602 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:55:14.475 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:55:14.475 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:55:14.632 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:55:16.477 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:55:16.640 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:55:16.640 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:55:16.640 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:55:16.640 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:55:17.081 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:55:17.688 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:55:19.318 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:55:19.693 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:55:19.973 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:55:19.973 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:55:19.973 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:55:19.973 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:55:19.973 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:55:20.177 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:55:20.222 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:55:20.258 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:55:20.294 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:55:20.337 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:55:20.360 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:55:20.387 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:55:20.414 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:55:20.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:55:20.482 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:55:20.485 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:55:20.485 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:55:20.485 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:55:20.485 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:55:20.485 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:55:20.485 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:55:20.485 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:55:20.486 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:55:20.486 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:55:20.486 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:55:20.486 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:55:20.486 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:55:20.486 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:55:20.753 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:55:21.562 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:55:21.562 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:55:22.858 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:55:23.563 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:55:23.570 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:55:24.008 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:55:25.449 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:55:25.996 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:55:26.024 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:55:26.048 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:55:26.074 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:55:26.101 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:55:26.127 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:55:26.153 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:55:26.180 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:55:26.208 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:55:26.236 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:55:26.272 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:55:26.275 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:55:26.275 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:55:26.275 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:55:26.275 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:55:26.275 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:55:26.275 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:55:26.276 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:55:26.276 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:55:26.276 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:55:26.276 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:55:26.276 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:55:26.276 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:55:26.277 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:55:26.372 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:55:26.372 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:55:26.372 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:55:26.372 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:55:27.082 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:55:27.082 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:55:27.495 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:55:29.074 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:55:29.083 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:55:29.609 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:55:29.609 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:55:29.609 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:55:29.609 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:55:29.609 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:55:30.212 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:55:31.648 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:55:31.833 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:55:31.857 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:55:31.880 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:55:31.924 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:55:31.950 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:55:31.976 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:55:32.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:55:32.028 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:55:32.053 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:55:32.089 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:55:32.120 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:55:32.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:55:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:55:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:55:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:55:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:55:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:55:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:55:32.123 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:55:32.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:55:32.124 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:55:32.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:55:32.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:55:32.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:55:32.852 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:55:32.852 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:55:33.228 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:55:33.857 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:55:34.853 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:55:35.181 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:55:36.276 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:55:37.034 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:55:37.035 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:55:37.985 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:55:39.499 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:55:40.118 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:55:40.118 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:55:40.862 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:55:42.345 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:55:42.873 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:55:42.874 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:55:43.768 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:55:45.384 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:55:45.957 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:55:45.957 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:55:47.076 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:55:47.694 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:55:47.731 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:55:47.763 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:55:47.808 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:55:47.854 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:55:47.906 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:55:47.948 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:55:47.981 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:55:48.012 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:55:48.041 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:55:48.094 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:55:48.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:55:48.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:55:48.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:55:48.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:55:48.097 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:55:48.097 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:55:48.097 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:55:48.097 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:55:48.097 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:55:48.097 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:55:48.098 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:55:48.098 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:55:48.098 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:55:48.167 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:55:48.719 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:55:48.720 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:55:48.876 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:55:48.877 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:55:49.766 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:55:50.721 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:55:51.124 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:55:51.950 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:55:51.950 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:55:52.867 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:55:53.984 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:55:54.007 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:55:54.030 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:55:54.082 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:55:54.122 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:55:54.159 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:55:54.194 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:55:54.228 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:55:54.260 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:55:54.289 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:55:54.316 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:55:54.317 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:55:54.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:55:54.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:55:54.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:55:54.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:55:54.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:55:54.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:55:54.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:55:54.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:55:54.319 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:55:54.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:55:54.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:55:54.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:55:54.818 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:55:54.888 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:55:54.888 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:55:55.244 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:55:55.245 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:55:55.536 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:55:56.889 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:55:57.688 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:55:58.407 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:55:58.407 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:55:59.881 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:56:00.030 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:56:00.059 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:56:00.083 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:56:00.107 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:56:00.133 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:56:00.160 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:56:00.187 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:56:00.211 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:56:00.238 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:56:00.269 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:56:00.294 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:56:00.295 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:56:00.295 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:56:00.295 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:56:00.295 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:56:00.296 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:56:00.296 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:56:00.296 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:56:00.296 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:56:00.296 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:56:00.296 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:56:00.296 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:56:00.297 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:56:00.297 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:56:00.968 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:56:00.968 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:56:01.203 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:56:01.712 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:56:01.712 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:56:02.595 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:56:02.970 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:56:04.114 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:56:04.852 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:56:04.853 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:56:05.909 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:56:06.381 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:56:06.413 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:56:06.438 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:56:06.461 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:56:06.487 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:56:06.513 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:56:06.539 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:56:06.565 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:56:06.592 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:56:06.619 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:56:06.645 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:56:06.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:56:06.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:56:06.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:56:06.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:56:06.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:56:06.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:56:06.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:56:06.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:56:06.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:56:06.648 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:56:06.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:56:06.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:56:06.649 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:56:07.351 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:56:07.351 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:56:07.702 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:56:08.257 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:56:08.258 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:56:08.811 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:56:09.353 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:56:10.845 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:56:11.598 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:56:11.598 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:56:12.533 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:56:12.573 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:56:12.608 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:56:12.638 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:56:12.671 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:56:12.698 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:56:12.725 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:56:12.752 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:56:12.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:56:12.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:56:12.831 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:56:12.834 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:56:12.834 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:56:12.834 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:56:12.835 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:56:12.835 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:56:12.835 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:56:12.835 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:56:12.835 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:56:12.835 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:56:12.835 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:56:12.836 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:56:12.836 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:56:12.836 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:56:12.875 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:56:13.634 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:56:13.634 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:56:14.353 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:56:14.873 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:56:14.874 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:56:15.635 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:56:15.896 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:56:17.693 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:56:18.399 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:56:18.400 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:56:18.813 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:56:19.670 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:56:19.715 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:56:19.753 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:56:19.785 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:56:19.815 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:56:19.842 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:56:19.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:56:19.882 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:56:19.912 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:56:19.951 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:56:19.984 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:56:19.986 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:56:19.986 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:56:19.986 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:56:19.986 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:56:19.987 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:56:19.987 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:56:19.987 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:56:19.987 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:56:19.987 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:56:19.987 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:56:19.988 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:56:19.988 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:56:19.988 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:56:20.537 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:56:20.832 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:56:20.832 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:56:21.156 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:56:21.157 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:56:22.478 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:56:22.833 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:56:23.042 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:56:23.814 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:56:23.815 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:56:25.913 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:56:26.218 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:56:26.838 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:56:27.079 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:56:27.126 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:56:27.166 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:56:27.204 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:56:27.237 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:56:27.272 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:56:27.301 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:56:27.330 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:56:27.362 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:56:27.384 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:56:27.385 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:56:27.389 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:56:27.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:56:27.392 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:56:27.392 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:56:27.392 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:56:27.392 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:56:27.392 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:56:27.392 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:56:27.393 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:56:27.393 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:56:27.393 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:56:27.393 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:56:27.393 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:56:27.393 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:56:27.898 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:56:27.898 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:56:28.588 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:56:29.429 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:56:29.899 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:56:30.065 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:56:30.066 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:56:32.763 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:56:32.861 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:56:32.885 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:56:32.888 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:56:32.911 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:56:32.934 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:56:32.960 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:56:32.986 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:56:33.013 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:56:33.039 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:56:33.065 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:56:33.091 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:56:33.118 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:56:33.120 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:56:33.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:56:33.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:56:33.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:56:33.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:56:33.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:56:33.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:56:33.121 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:56:33.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:56:33.122 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:56:33.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:56:33.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:56:33.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:56:33.469 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:56:33.469 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:56:33.938 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:56:33.940 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:56:35.554 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:56:35.769 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:56:35.941 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:56:36.052 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:56:36.052 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:56:38.974 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:56:39.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:56:39.009 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:56:39.027 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:56:39.052 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:56:39.077 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:56:39.078 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:56:39.105 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:56:39.131 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:56:39.158 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:56:39.202 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:56:39.233 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:56:39.268 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:56:39.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:56:39.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:56:39.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:56:39.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:56:39.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:56:39.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:56:39.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:56:39.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:56:39.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:56:39.272 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:56:39.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:56:39.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:56:39.273 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:56:39.697 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:56:39.698 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:56:40.159 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:56:40.160 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:56:41.794 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:56:41.794 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:56:41.794 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:56:41.794 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:56:41.983 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:56:42.160 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:56:42.303 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:56:45.061 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:56:45.084 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:56:45.108 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:56:45.132 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:56:45.158 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:56:45.184 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:56:45.210 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:56:45.236 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:56:45.263 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:56:45.290 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:56:45.316 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:56:45.317 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:56:45.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:56:45.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:56:45.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:56:45.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:56:45.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:56:45.318 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:56:45.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:56:45.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:56:45.319 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:56:45.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:56:45.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:56:45.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:56:45.368 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:56:45.904 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:56:46.131 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:56:46.132 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:56:46.137 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:56:48.133 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:56:48.222 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:56:48.519 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:56:48.790 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:56:51.724 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:56:51.826 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:56:51.826 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:56:51.826 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:56:51.826 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:56:51.894 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:56:52.475 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:56:52.575 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:56:52.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:56:52.660 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:56:52.693 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:56:52.725 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:56:52.751 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:56:52.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:56:52.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:56:52.833 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:56:52.861 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:56:52.862 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:56:52.862 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:56:52.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:56:52.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:56:52.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:56:52.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:56:52.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:56:52.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:56:52.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:56:52.864 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:56:52.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:56:52.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:56:52.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:56:53.418 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:56:53.419 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:56:54.644 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:56:54.650 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:56:54.650 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:56:54.651 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:56:54.651 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:56:54.651 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:56:55.276 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:56:55.420 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:56:57.918 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:56:58.079 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:56:58.080 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:56:58.080 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:56:58.080 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:56:58.080 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:56:58.684 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:56:58.976 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:56:59.544 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:56:59.582 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:56:59.611 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:56:59.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:56:59.666 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:56:59.709 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:56:59.736 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:56:59.763 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:56:59.791 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:56:59.826 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:56:59.828 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:56:59.828 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:56:59.828 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:56:59.829 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:56:59.829 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:56:59.829 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:56:59.829 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:56:59.829 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:56:59.829 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:56:59.829 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:56:59.830 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:56:59.830 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:56:59.830 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:57:00.547 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:57:00.548 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:57:00.600 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:57:01.078 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:57:01.797 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:57:02.548 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:57:03.978 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:57:04.331 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:57:05.127 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:57:05.527 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:57:05.548 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:57:05.571 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:57:05.594 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:57:05.620 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:57:05.646 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:57:05.672 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:57:05.698 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:57:05.724 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:57:05.752 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:57:05.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:57:05.780 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:57:05.780 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:57:05.780 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:57:05.781 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:57:05.781 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:57:05.781 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:57:05.781 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:57:05.781 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:57:05.781 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:57:05.782 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:57:05.782 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:57:05.782 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:57:05.782 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:57:06.320 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:57:06.320 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:57:06.852 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:57:07.241 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:57:07.241 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:57:07.242 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:57:07.242 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:57:07.242 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:57:07.944 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:57:08.321 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:57:09.919 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:57:10.887 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:57:10.888 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:57:10.888 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:57:10.888 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:57:11.392 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:57:11.404 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:57:11.419 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:57:11.442 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:57:11.465 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:57:11.491 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:57:11.518 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:57:11.549 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:57:11.575 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:57:11.601 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:57:11.644 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:57:11.670 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:57:11.671 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:57:11.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:57:11.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:57:11.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:57:11.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:57:11.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:57:11.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:57:11.672 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:57:11.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:57:11.673 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:57:11.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:57:11.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:57:11.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:57:12.253 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:57:12.253 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:57:12.626 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:57:13.925 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:57:14.254 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:57:14.683 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:57:15.939 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:57:17.364 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:57:17.388 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:57:17.413 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:57:17.436 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:57:17.462 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:57:17.488 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:57:17.514 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:57:17.519 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:57:17.519 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:57:17.520 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:57:17.520 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:57:17.520 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:57:17.541 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:57:17.567 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:57:17.593 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:57:17.620 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:57:17.622 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:57:17.622 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:57:17.622 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:57:17.622 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:57:17.622 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:57:17.622 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:57:17.623 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:57:17.623 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:57:17.623 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:57:17.623 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:57:17.623 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:57:17.623 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:57:17.623 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:57:18.165 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:57:18.365 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:57:18.365 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:57:18.859 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:57:20.367 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:57:20.457 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:57:21.125 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:57:21.125 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:57:21.928 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:57:23.644 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:57:23.880 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:57:24.425 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:57:24.425 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:57:24.496 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:57:24.546 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:57:24.592 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:57:24.628 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:57:24.660 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:57:24.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:57:24.718 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:57:24.745 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:57:24.773 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:57:24.799 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:57:24.801 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:57:24.801 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:57:24.801 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:57:24.801 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:57:24.802 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:57:24.802 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:57:24.802 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:57:24.802 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:57:24.803 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:57:24.803 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:57:24.803 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:57:24.803 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:57:24.804 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:57:24.863 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:57:25.576 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:57:25.577 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:57:26.758 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:57:27.414 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:57:27.414 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:57:27.580 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:57:27.960 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:57:29.989 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:57:30.505 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:57:30.535 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:57:30.551 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:57:30.552 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:57:30.565 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:57:30.589 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:57:30.615 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:57:30.641 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:57:30.673 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:57:30.705 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:57:30.736 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:57:30.766 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:57:30.794 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:57:30.795 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:57:30.795 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:57:30.796 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:57:30.796 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:57:30.796 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:57:30.796 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:57:30.796 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:57:30.796 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:57:30.797 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:57:30.797 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:57:30.797 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:57:30.797 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:57:30.797 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:57:30.851 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:57:31.538 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:57:31.539 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:57:33.205 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:57:33.540 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:57:33.707 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:57:33.708 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:57:34.098 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:57:36.071 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:57:36.096 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:57:36.137 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:57:36.172 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:57:36.206 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:57:36.237 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:57:36.267 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:57:36.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:57:36.323 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:57:36.350 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:57:36.352 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:57:36.376 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:57:36.377 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:57:36.378 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:57:36.378 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:57:36.378 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:57:36.378 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:57:36.378 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:57:36.378 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:57:36.378 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:57:36.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:57:36.379 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:57:36.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:57:36.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:57:36.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:57:36.742 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:57:36.968 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:57:36.969 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:57:37.113 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:57:37.113 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:57:39.114 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:57:39.339 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:57:39.978 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:57:39.979 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:57:40.094 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:57:42.145 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:57:42.180 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:57:42.211 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:57:42.239 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:57:42.266 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:57:42.292 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:57:42.318 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:57:42.344 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:57:42.371 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:57:42.378 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:57:42.398 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:57:42.424 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:57:42.426 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:57:42.426 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:57:42.426 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:57:42.426 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:57:42.427 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:57:42.427 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:57:42.427 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:57:42.427 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:57:42.427 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:57:42.427 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:57:42.427 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:57:42.428 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:57:42.428 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:57:43.033 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:57:43.033 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:57:43.049 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:57:43.194 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:57:43.194 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:57:45.195 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:57:45.248 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:57:45.848 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:57:45.849 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:57:46.042 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:57:48.126 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:57:48.151 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:57:48.174 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:57:48.197 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:57:48.223 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:57:48.249 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:57:48.276 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:57:48.302 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:57:48.328 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:57:48.352 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:57:48.354 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:57:48.380 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:57:48.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:57:48.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:57:48.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:57:48.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:57:48.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:57:48.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:57:48.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:57:48.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:57:48.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:57:48.382 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:57:48.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:57:48.383 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:57:48.383 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:57:48.820 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:57:48.999 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:57:49.000 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:57:49.196 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:57:49.197 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:57:51.198 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:57:51.218 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:57:51.833 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:57:51.915 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:57:52.157 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:57:54.187 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:57:54.215 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:57:54.239 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:57:54.263 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:57:54.288 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:57:54.314 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:57:54.340 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:57:54.367 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:57:54.372 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:57:54.394 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:57:54.421 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:57:54.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:57:54.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:57:54.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:57:54.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:57:54.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:57:54.449 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:57:54.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:57:54.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:57:54.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:57:54.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:57:54.450 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:57:54.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:57:54.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:57:54.450 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:57:55.023 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:57:55.024 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:57:55.170 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:57:55.183 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:57:55.183 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:57:57.184 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:57:57.329 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:57:57.973 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:57:57.981 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:57:57.981 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:58:00.294 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:58:00.315 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:58:00.339 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:58:00.362 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:58:00.389 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:58:00.415 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:58:00.442 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:58:00.468 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:58:00.495 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:58:00.522 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:58:00.537 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:58:00.558 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:58:00.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:58:00.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:58:00.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:58:00.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:58:00.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:58:00.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:58:00.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:58:00.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:58:00.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:58:00.561 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:58:00.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:58:00.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:58:00.563 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:58:00.686 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:58:01.201 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:58:01.201 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:58:01.428 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:58:01.429 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:58:03.429 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:58:03.554 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:58:04.075 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:58:04.076 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:58:06.309 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:58:06.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:58:06.363 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:58:06.392 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:58:06.421 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:58:06.448 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:58:06.474 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:58:06.500 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:58:06.523 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:58:06.526 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:58:06.552 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:58:06.578 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:58:06.579 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:58:06.579 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:58:06.580 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:58:06.580 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:58:06.580 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:58:06.580 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:58:06.580 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:58:06.580 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:58:06.580 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:58:06.581 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:58:06.581 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:58:06.581 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:58:06.581 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:58:07.162 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:58:07.162 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:58:07.309 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:58:07.309 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:58:09.130 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:58:09.310 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:58:09.669 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:58:09.669 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:58:11.859 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:58:11.881 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:58:11.906 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:58:11.915 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:58:11.929 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:58:11.957 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:58:11.981 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:58:12.008 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:58:12.033 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:58:12.060 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:58:12.086 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:58:12.112 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:58:12.114 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:58:12.114 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:58:12.114 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:58:12.114 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:58:12.114 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:58:12.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:58:12.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:58:12.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:58:12.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:58:12.115 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:58:12.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:58:12.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:58:12.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:58:12.442 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:58:12.442 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:58:12.759 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:58:12.760 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:58:13.596 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:58:14.672 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:58:14.760 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:58:15.304 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:58:15.305 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:58:16.133 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:58:17.203 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:58:17.236 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:58:17.261 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:58:17.287 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:58:17.333 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:58:17.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:58:17.406 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:58:17.427 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:58:17.455 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:58:17.482 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:58:17.512 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:58:17.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:58:17.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:58:17.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:58:17.515 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:58:17.515 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:58:17.515 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:58:17.515 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:58:17.515 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:58:17.515 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:58:17.515 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:58:17.516 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:58:17.516 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:58:17.516 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:58:18.286 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:58:18.477 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:58:18.478 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:58:18.926 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:58:18.927 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-22T23:58:19.961 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:58:20.478 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:58:21.125 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:58:21.672 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:58:21.672 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-22T23:58:22.732 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:58:23.101 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:58:23.136 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:58:23.180 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:58:23.203 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:58:23.230 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:58:23.256 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:58:23.283 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:58:23.314 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:58:23.337 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:58:23.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:58:23.391 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:58:23.392 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:58:23.393 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:58:23.393 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:58:23.393 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:58:23.393 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:58:23.393 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:58:23.394 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:58:23.394 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:58:23.394 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:58:23.394 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:58:23.394 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:58:23.394 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:58:23.394 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:58:24.099 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:58:24.100 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:58:24.494 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:58:25.118 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-22T23:58:25.982 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:58:26.100 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:58:27.731 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:58:28.365 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-22T23:58:28.696 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:58:28.719 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:58:28.742 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:58:28.765 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:58:28.792 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:58:28.819 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:58:28.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:58:28.873 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:58:28.884 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:58:28.907 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:58:28.930 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:58:28.959 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:58:28.962 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:58:28.962 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:58:28.962 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:58:28.962 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:58:28.962 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:58:28.962 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:58:28.963 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:58:28.963 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:58:28.963 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:58:28.963 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:58:28.963 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:58:28.963 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:58:28.963 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:58:30.133 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:58:30.133 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:58:31.073 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:58:31.074 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:58:31.074 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:58:31.074 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:58:31.074 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:58:31.635 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-22T23:58:32.135 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:58:32.167 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:58:34.360 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:58:34.360 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:58:34.361 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:58:34.361 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:58:34.797 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:58:34.972 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:58:34.995 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:58:35.024 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:58:35.042 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-22T23:58:35.056 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:58:35.079 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:58:35.108 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:58:35.135 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:58:35.160 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:58:35.186 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:58:35.213 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:58:35.239 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:58:35.240 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:58:35.241 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:58:35.241 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:58:35.241 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:58:35.241 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:58:35.241 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:58:35.242 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:58:35.242 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:58:35.242 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:58:35.242 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:58:35.243 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:58:35.243 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:58:35.244 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:58:36.100 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:58:36.101 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:58:37.680 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:58:37.731 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:58:38.102 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:58:38.498 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-22T23:58:40.356 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:58:40.486 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:58:40.517 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:58:40.554 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:58:40.577 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:58:40.605 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:58:40.632 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:58:40.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:58:40.686 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:58:40.714 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:58:40.741 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:58:40.759 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:58:40.759 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-22T23:58:40.759 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-22T23:58:40.760 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-22T23:58:40.760 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:58:40.768 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:58:40.769 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:58:40.770 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:58:40.770 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:58:40.770 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:58:40.770 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:58:40.770 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:58:40.770 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:58:40.770 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:58:40.771 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:58:40.771 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:58:40.771 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:58:40.771 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:58:40.771 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:58:41.597 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-22T23:58:41.917 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:58:41.918 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:58:43.716 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:58:43.807 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:58:43.807 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:58:43.807 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:58:43.807 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:58:43.920 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:58:44.599 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-22T23:58:46.337 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:58:46.381 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:58:47.037 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:58:47.037 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:58:47.037 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:58:47.038 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:58:47.038 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:58:47.245 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:58:47.271 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:58:47.308 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:58:47.343 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:58:47.376 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:58:47.405 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:58:47.434 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:58:47.461 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:58:47.491 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:58:47.514 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:58:47.516 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:58:47.516 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:58:47.516 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:58:47.516 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:58:47.516 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:58:47.517 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:58:47.517 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:58:47.517 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:58:47.517 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:58:47.517 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:58:47.517 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:58:47.517 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:58:47.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:58:47.776 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-22T23:58:48.250 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:58:48.251 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:58:49.979 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:58:50.045 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:58:50.251 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:58:50.827 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-22T23:58:52.656 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:58:52.844 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:58:52.870 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:58:52.896 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:58:52.921 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:58:52.948 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:58:52.974 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:58:53.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:58:53.029 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:58:53.058 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:58:53.060 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:58:53.060 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:58:53.060 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-22T23:58:53.061 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:58:53.085 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:58:53.111 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:58:53.113 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:58:53.113 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:58:53.113 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:58:53.114 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:58:53.114 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:58:53.114 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:58:53.114 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:58:53.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:58:53.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:58:53.115 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:58:53.115 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:58:53.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:58:53.116 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:58:53.918 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:58:53.919 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:58:54.068 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-22T23:58:55.919 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:58:55.990 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:58:56.513 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-22T23:58:56.514 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-22T23:58:56.514 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-22T23:58:56.514 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-22T23:58:56.514 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-22T23:58:57.131 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-22T23:58:58.535 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:58:58.854 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:58:58.900 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:58:58.933 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:58:58.963 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:58:58.991 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:58:59.018 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:58:59.045 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:58:59.072 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:58:59.098 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:58:59.125 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:58:59.164 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:58:59.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:58:59.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:58:59.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:58:59.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:58:59.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:58:59.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:58:59.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:58:59.167 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:58:59.167 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:58:59.167 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:58:59.167 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:58:59.167 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:58:59.167 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:58:59.873 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:58:59.873 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:59:00.377 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-22T23:59:00.937 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-22T23:59:01.875 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:59:02.123 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:59:03.566 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:59:04.047 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:59:04.048 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-22T23:59:04.581 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:59:04.605 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:59:04.637 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:59:04.665 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:59:04.694 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:59:04.722 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:59:04.747 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:59:04.765 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:59:04.785 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:59:04.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:59:04.834 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:59:04.861 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:59:04.862 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:59:04.862 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:59:04.862 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:59:04.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:59:04.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:59:04.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:59:04.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:59:04.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:59:04.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:59:04.863 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:59:04.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:59:04.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:59:04.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:59:05.747 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:59:05.747 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:59:06.610 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:59:07.218 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:59:07.219 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-22T23:59:07.748 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:59:07.794 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:59:09.741 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:59:10.392 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:59:10.439 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:59:10.440 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-22T23:59:10.757 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:59:10.780 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:59:10.804 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:59:10.828 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:59:10.860 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:59:10.889 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:59:10.912 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:59:10.938 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:59:10.966 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:59:10.994 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:59:11.041 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:59:11.042 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:59:11.042 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:59:11.043 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:59:11.043 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:59:11.043 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:59:11.043 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:59:11.043 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:59:11.043 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:59:11.043 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:59:11.044 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:59:11.044 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:59:11.044 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:59:11.044 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:59:11.722 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:59:11.722 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:59:13.095 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:59:13.596 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:59:13.597 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-22T23:59:13.723 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:59:14.002 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:59:16.103 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:59:16.622 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:59:16.639 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:59:16.639 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-22T23:59:16.651 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:59:16.682 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:59:16.711 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:59:16.740 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:59:16.767 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:59:16.786 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:59:16.794 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:59:16.820 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:59:16.848 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:59:16.877 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:59:16.903 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:59:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:59:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:59:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:59:16.905 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:59:16.906 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:59:16.906 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:59:16.906 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:59:16.906 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:59:16.906 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:59:16.906 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:59:16.906 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:59:16.907 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:59:16.907 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:59:17.775 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:59:17.775 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:59:19.148 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:59:19.776 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:59:19.868 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:59:19.868 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-22T23:59:20.998 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:59:22.340 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:59:22.364 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:59:22.387 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:59:22.412 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:59:22.438 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:59:22.464 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:59:22.491 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:59:22.493 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:59:22.524 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:59:22.549 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:59:22.577 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:59:22.603 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:59:22.605 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:59:22.605 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:59:22.605 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:59:22.605 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:59:22.606 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:59:22.606 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:59:22.606 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:59:22.606 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:59:22.606 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:59:22.606 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:59:22.607 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:59:22.607 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:59:22.607 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:59:23.143 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:59:23.144 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-22T23:59:23.292 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:59:23.292 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:59:23.918 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:59:25.165 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:59:25.293 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:59:25.844 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:59:25.844 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-22T23:59:27.122 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:59:28.401 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:59:28.646 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:59:29.160 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:59:29.161 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-22T23:59:29.257 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:59:29.303 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:59:29.344 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:59:29.379 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:59:29.411 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:59:29.441 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:59:29.470 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:59:29.497 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:59:29.524 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:59:29.551 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:59:29.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:59:29.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:59:29.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:59:29.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:59:29.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:59:29.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:59:29.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:59:29.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:59:29.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:59:29.555 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:59:29.556 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:59:29.556 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:59:29.556 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:59:29.962 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:59:30.217 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:59:30.217 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:59:31.655 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:59:32.218 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:59:32.234 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:59:32.234 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-22T23:59:33.144 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:59:34.898 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:59:34.923 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:59:34.955 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:59:34.982 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:59:35.023 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:59:35.059 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:59:35.093 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:59:35.093 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:59:35.119 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:59:35.146 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:59:35.173 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:59:35.199 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:59:35.200 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:59:35.200 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:59:35.200 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:59:35.200 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:59:35.201 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:59:35.201 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:59:35.201 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:59:35.201 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:59:35.201 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:59:35.201 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:59:35.201 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:59:35.202 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:59:35.202 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:59:35.821 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:59:35.822 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-22T23:59:35.979 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:59:35.979 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:59:35.985 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-22T23:59:37.980 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:59:38.061 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:59:38.705 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:59:38.705 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-22T23:59:38.998 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-22T23:59:40.981 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:59:41.006 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:59:41.031 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:59:41.057 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:59:41.087 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:59:41.114 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:59:41.141 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:59:41.168 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:59:41.197 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:59:41.224 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:59:41.241 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:59:41.250 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:59:41.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:59:41.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:59:41.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:59:41.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:59:41.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:59:41.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:59:41.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:59:41.252 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:59:41.253 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:59:41.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:59:41.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:59:41.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:59:41.910 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:59:41.910 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-22T23:59:42.020 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-22T23:59:42.085 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:59:42.085 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:59:44.086 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:59:44.225 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:59:44.932 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:59:44.933 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-22T23:59:45.149 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-22T23:59:47.102 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:59:47.135 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:59:47.165 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:59:47.192 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:59:47.218 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:59:47.245 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:59:47.271 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:59:47.298 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:59:47.324 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:59:47.341 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:59:47.351 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:59:47.378 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:59:47.379 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-22T23:59:47.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-22T23:59:47.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-22T23:59:47.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-22T23:59:47.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-22T23:59:47.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-22T23:59:47.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-22T23:59:47.380 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-22T23:59:47.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-22T23:59:47.381 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:59:47.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:59:47.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-22T23:59:47.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-22T23:59:47.942 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-22T23:59:47.950 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:59:47.951 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-22T23:59:48.098 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:59:48.098 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:59:50.099 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-22T23:59:50.137 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:59:50.732 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:59:50.732 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-22T23:59:51.019 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-22T23:59:53.039 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:59:53.283 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:59:53.715 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-22T23:59:53.783 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:59:53.783 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-22T23:59:53.893 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:59:53.950 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:59:54.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:59:54.030 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:59:54.061 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:59:54.085 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:59:54.110 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:59:54.137 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:59:54.166 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-22T23:59:54.193 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-22T23:59:54.195 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-22T23:59:54.195 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-22T23:59:54.195 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-22T23:59:54.195 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-22T23:59:54.195 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-22T23:59:54.195 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-22T23:59:54.196 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-22T23:59:54.196 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-22T23:59:54.196 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-22T23:59:54.196 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-22T23:59:54.196 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-22T23:59:54.196 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-22T23:59:54.196 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-22T23:59:54.964 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-22T23:59:54.964 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-22T23:59:56.083 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:59:56.693 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:59:56.693 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-22T23:59:56.965 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-22T23:59:58.078 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-22T23:59:59.332 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-22T23:59:59.619 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-22T23:59:59.656 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-22T23:59:59.707 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-22T23:59:59.748 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-22T23:59:59.785 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-22T23:59:59.820 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-22T23:59:59.854 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-22T23:59:59.886 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-22T23:59:59.926 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-22T23:59:59.927 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-22T23:59:59.935 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-22T23:59:59.981 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:00:00.023 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:00:00.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:00:00.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:00:00.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:00:00.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:00:00.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:00:00.025 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:00:00.025 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:00:00.025 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:00:00.025 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:00:00.025 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:00:00.025 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:00:00.025 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:00:00.026 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:00:00.485 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:00:00.486 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:00:00.943 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:00:02.476 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:00:02.486 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:00:03.246 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:00:03.246 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-23T00:00:04.136 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:00:05.517 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:00:05.542 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:00:05.564 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:00:05.587 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:00:05.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:00:05.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:00:05.655 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:00:05.666 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:00:05.705 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:00:05.719 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:00:05.746 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:00:05.772 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:00:05.773 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:00:05.774 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:00:05.774 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:00:05.774 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:00:05.774 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:00:05.774 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:00:05.774 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:00:05.775 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:00:05.775 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:00:05.775 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:00:05.775 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:00:05.775 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:00:05.775 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:00:06.373 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:00:06.373 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-23T00:00:06.537 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:00:06.538 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:00:06.953 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:00:08.539 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:00:08.774 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:00:08.775 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:00:08.775 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:00:08.775 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:00:09.628 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-23T00:00:10.025 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:00:11.767 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:00:11.797 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:00:11.825 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:00:11.848 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:00:11.874 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:00:11.901 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:00:11.927 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:00:11.954 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:00:11.980 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:00:12.004 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:00:12.006 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:00:12.033 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:00:12.035 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:00:12.035 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:00:12.035 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:00:12.035 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:00:12.035 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:00:12.035 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:00:12.036 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:00:12.036 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:00:12.036 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:00:12.036 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:00:12.036 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:00:12.036 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:00:12.036 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:00:12.642 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-23T00:00:12.794 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:00:12.795 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:00:12.851 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:00:14.795 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:00:14.913 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:00:15.609 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-23T00:00:16.044 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:00:18.015 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:00:18.292 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:00:18.292 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:00:18.292 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:00:18.292 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:00:18.848 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-23T00:00:18.867 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:00:18.915 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:00:18.955 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:00:18.992 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:00:19.043 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:00:19.065 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:00:19.090 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:00:19.132 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:00:19.182 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:00:19.230 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:00:19.269 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:00:19.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:00:19.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:00:19.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:00:19.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:00:19.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:00:19.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:00:19.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:00:19.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:00:19.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:00:19.272 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:00:19.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:00:19.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:00:19.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:00:20.097 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:00:20.098 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:00:21.660 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:00:21.661 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:00:21.661 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:00:21.661 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:00:21.661 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:00:22.035 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:00:22.099 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:00:22.183 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-23T00:00:24.593 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:00:24.780 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:00:24.780 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:00:24.780 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:00:24.780 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:00:24.781 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:00:24.998 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:00:25.018 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:00:25.045 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:00:25.081 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:00:25.116 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:00:25.149 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:00:25.184 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:00:25.214 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:00:25.244 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:00:25.272 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:00:25.300 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:00:25.301 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:00:25.301 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:00:25.301 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:00:25.301 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:00:25.301 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:00:25.301 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:00:25.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:00:25.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:00:25.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:00:25.302 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:00:25.302 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:00:25.303 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:00:25.303 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:00:25.540 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-23T00:00:25.998 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:00:25.998 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:00:27.917 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:00:28.000 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:00:28.053 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:00:28.608 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-23T00:00:30.723 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:00:30.753 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:00:30.783 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:00:30.810 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:00:30.838 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:00:30.865 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:00:30.891 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:00:30.900 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:00:30.901 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:00:30.901 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:00:30.901 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:00:30.901 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:00:30.917 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:00:30.944 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:00:30.952 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:00:30.972 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:00:31.014 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:00:31.015 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:00:31.015 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:00:31.015 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:00:31.016 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:00:31.016 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:00:31.016 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:00:31.016 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:00:31.016 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:00:31.017 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:00:31.017 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:00:31.017 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:00:31.017 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:00:31.017 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:00:31.454 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-23T00:00:31.956 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:00:31.956 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:00:33.958 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:00:34.032 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:00:34.043 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:00:34.044 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:00:34.044 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:00:34.044 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:00:34.044 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:00:34.972 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-23T00:00:36.696 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:00:36.744 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:00:36.780 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:00:36.803 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:00:36.830 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:00:36.886 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:00:36.936 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:00:36.981 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:00:37.027 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:00:37.066 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:00:37.096 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:00:37.126 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:00:37.128 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:00:37.128 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:00:37.128 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:00:37.128 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:00:37.128 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:00:37.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:00:37.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:00:37.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:00:37.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:00:37.129 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:00:37.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:00:37.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:00:37.129 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:00:37.396 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:00:37.789 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:00:37.790 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:00:38.196 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-23T00:00:39.791 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:00:40.042 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:00:40.143 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:00:41.022 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-23T00:00:42.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:00:42.582 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:00:42.629 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:00:42.656 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:00:42.684 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:00:42.710 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:00:42.737 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:00:42.764 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:00:42.792 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:00:42.821 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:00:42.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:00:42.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:00:42.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:00:42.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:00:42.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:00:42.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:00:42.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:00:42.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:00:42.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:00:42.850 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:00:42.850 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:00:42.851 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:00:42.851 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:00:42.851 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:00:42.853 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:00:43.403 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:00:43.403 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:00:43.403 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:00:43.404 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:00:43.404 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:00:43.524 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:00:43.524 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:00:44.160 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-23T00:00:45.525 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:00:46.054 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:00:46.207 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:00:47.058 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:00:47.059 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-23T00:00:48.464 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:00:48.499 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:00:48.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:00:48.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:00:48.584 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:00:48.611 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:00:48.637 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:00:48.665 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:00:48.695 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:00:48.728 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:00:48.759 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:00:48.761 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:00:48.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:00:48.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:00:48.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:00:48.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:00:48.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:00:48.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:00:48.762 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:00:48.763 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:00:48.763 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:00:48.763 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:00:48.763 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:00:48.763 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:00:48.893 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:00:49.181 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:00:49.312 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:00:49.313 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:00:49.782 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:00:49.782 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-23T00:00:51.314 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:00:52.142 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:00:52.379 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:00:53.413 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:00:53.414 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-23T00:00:54.040 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:00:54.065 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:00:54.097 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:00:54.121 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:00:54.169 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:00:54.199 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:00:54.226 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:00:54.253 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:00:54.280 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:00:54.309 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:00:54.337 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:00:54.338 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:00:54.339 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:00:54.339 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:00:54.339 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:00:54.339 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:00:54.339 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:00:54.339 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:00:54.339 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:00:54.340 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:00:54.340 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:00:54.340 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:00:54.340 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:00:54.340 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:00:55.123 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:00:55.124 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:00:55.166 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:00:55.505 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:00:56.185 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:00:56.185 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-23T00:00:57.125 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:00:58.144 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:00:58.708 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:00:59.400 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:00:59.401 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-23T00:00:59.961 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:00:59.984 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:01:00.008 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:01:00.037 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:01:00.063 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:01:00.089 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:01:00.115 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:01:00.143 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:01:00.172 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:01:00.200 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:01:00.231 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:01:00.231 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:01:00.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:01:00.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:01:00.232 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:01:00.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:01:00.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:01:00.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:01:00.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:01:00.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:01:00.234 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:01:00.234 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:01:00.234 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:01:00.234 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:01:00.912 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:01:00.989 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:01:00.990 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:01:01.561 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:01:02.147 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:01:02.147 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-23T00:01:02.991 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:01:04.047 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:01:04.895 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:01:05.565 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:01:05.566 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-23T00:01:05.954 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:01:05.978 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:01:06.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:01:06.027 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:01:06.053 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:01:06.079 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:01:06.105 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:01:06.131 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:01:06.158 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:01:06.186 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:01:06.212 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:01:06.215 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:01:06.216 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:01:06.216 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:01:06.216 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:01:06.216 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:01:06.216 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:01:06.216 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:01:06.216 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:01:06.217 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:01:06.217 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:01:06.217 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:01:06.217 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:01:06.217 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:01:06.986 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:01:06.986 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:01:07.050 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:01:07.849 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:01:08.464 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:01:08.464 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-23T00:01:08.990 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:01:10.065 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:01:11.340 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:01:11.633 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:01:11.673 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:01:11.709 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:01:11.741 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:01:11.769 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:01:11.796 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:01:11.823 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:01:11.849 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:01:11.876 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:01:11.903 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:01:11.931 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:01:11.933 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:01:11.933 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-23T00:01:11.934 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:01:11.934 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:01:11.935 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:01:11.935 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:01:11.935 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:01:11.935 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:01:11.935 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:01:11.935 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:01:11.935 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:01:11.936 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:01:11.936 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:01:11.936 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:01:11.936 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:01:12.516 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:01:12.516 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:01:12.899 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:01:14.172 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:01:14.517 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:01:15.107 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:01:15.107 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-23T00:01:16.071 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:01:17.826 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:01:17.920 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:01:17.943 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:01:17.972 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:01:17.990 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:01:18.016 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:01:18.041 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:01:18.067 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:01:18.094 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:01:18.120 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:01:18.166 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:01:18.205 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:01:18.207 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:01:18.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:01:18.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:01:18.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:01:18.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:01:18.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:01:18.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:01:18.208 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:01:18.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:01:18.209 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:01:18.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:01:18.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:01:18.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:01:18.315 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:01:18.316 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-23T00:01:18.760 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:01:18.760 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:01:19.090 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:01:20.694 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:01:20.761 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:01:21.406 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:01:21.407 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-23T00:01:22.075 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:01:23.959 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:01:24.013 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:01:24.533 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:01:24.534 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-23T00:01:24.813 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:01:24.838 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:01:24.879 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:01:24.915 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:01:24.966 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:01:25.008 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:01:25.046 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:01:25.084 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:01:25.088 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:01:25.123 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:01:25.157 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:01:25.159 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:01:25.159 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:01:25.159 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:01:25.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:01:25.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:01:25.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:01:25.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:01:25.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:01:25.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:01:25.161 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:01:25.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:01:25.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:01:25.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:01:25.864 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:01:25.864 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:01:27.059 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:01:27.659 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:01:27.659 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-23T00:01:27.866 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:01:29.077 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:01:30.391 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:01:30.542 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:01:30.567 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:01:30.590 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:01:30.613 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:01:30.638 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:01:30.664 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:01:30.691 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:01:30.717 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:01:30.743 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:01:30.769 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:01:30.795 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:01:30.797 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:01:30.797 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:01:30.797 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:01:30.797 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:01:30.798 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:01:30.798 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:01:30.798 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:01:30.798 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:01:30.798 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:01:30.798 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:01:30.798 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:01:30.799 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:01:30.799 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:01:30.925 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:01:30.925 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-23T00:01:31.299 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:01:31.299 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:01:31.867 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:01:33.188 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:01:33.299 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:01:33.803 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:01:33.803 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-23T00:01:35.165 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:01:36.251 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:01:36.279 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:01:36.302 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:01:36.325 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:01:36.352 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:01:36.378 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:01:36.405 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:01:36.431 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:01:36.458 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:01:36.484 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:01:36.490 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:01:36.511 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:01:36.512 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:01:36.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:01:36.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:01:36.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:01:36.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:01:36.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:01:36.513 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:01:36.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:01:36.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:01:36.514 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:01:36.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:01:36.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:01:36.514 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:01:37.131 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:01:37.131 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-23T00:01:37.289 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:01:37.289 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:01:38.043 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:01:39.291 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:01:39.453 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:01:40.426 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:01:40.427 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-23T00:01:41.141 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:01:42.448 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:01:42.480 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:01:42.504 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:01:42.530 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:01:42.556 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:01:42.582 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:01:42.609 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:01:42.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:01:42.668 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:01:42.697 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:01:42.727 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:01:42.729 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:01:42.729 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:01:42.730 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:01:42.730 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:01:42.730 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:01:42.730 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:01:42.730 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:01:42.730 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:01:42.731 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:01:42.731 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:01:42.731 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:01:42.731 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:01:42.731 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:01:43.273 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:01:43.273 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:01:43.304 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:01:43.701 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:01:43.702 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-23T00:01:43.723 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:01:45.275 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:01:46.420 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:01:47.228 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:01:47.228 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:01:47.229 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-23T00:01:48.194 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:01:48.220 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:01:48.244 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:01:48.267 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:01:48.295 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:01:48.321 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:01:48.348 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:01:48.377 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:01:48.404 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:01:48.431 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:01:48.457 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:01:48.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:01:48.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:01:48.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:01:48.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:01:48.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:01:48.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:01:48.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:01:48.461 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:01:48.461 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:01:48.461 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:01:48.461 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:01:48.461 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:01:48.462 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:01:49.173 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:01:49.173 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:01:49.607 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:01:49.895 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:01:50.123 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:01:50.127 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-23T00:01:51.174 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:01:52.681 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:01:53.173 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:01:53.576 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-23T00:01:54.234 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:01:55.091 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:01:55.127 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:01:55.155 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:01:55.183 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:01:55.210 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:01:55.236 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:01:55.262 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:01:55.289 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:01:55.315 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:01:55.343 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:01:55.345 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:01:55.345 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:01:55.345 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:01:55.345 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:01:55.345 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:01:55.345 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:01:55.345 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:01:55.346 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:01:55.346 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:01:55.346 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:01:55.346 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:01:55.346 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:01:55.346 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:01:55.824 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:01:55.824 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:01:55.824 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:01:55.824 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:01:55.824 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:01:55.845 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:01:55.845 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:01:55.954 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:01:56.324 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-23T00:01:57.846 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:01:58.473 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:01:58.473 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:01:58.474 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:01:58.474 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:01:58.474 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:01:58.791 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:01:59.272 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-23T00:02:00.529 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:02:00.552 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:02:00.577 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:02:00.602 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:02:00.628 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:02:00.654 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:02:00.681 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:02:00.727 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:02:00.768 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:02:00.806 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:02:00.836 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:02:00.838 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:02:00.838 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:02:00.838 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:02:00.838 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:02:00.838 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:02:00.839 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:02:00.839 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:02:00.839 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:02:00.839 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:02:00.839 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:02:00.839 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:02:00.839 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:02:00.840 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:02:01.482 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:02:01.482 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:02:01.544 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:02:01.720 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:02:02.382 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-23T00:02:03.483 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:02:04.918 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:02:05.107 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:02:05.733 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-23T00:02:06.483 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:02:06.510 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:02:06.534 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:02:06.561 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:02:06.588 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:02:06.615 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:02:06.641 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:02:06.669 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:02:06.699 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:02:06.727 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:02:06.755 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:02:06.756 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:02:06.756 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:02:06.756 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:02:06.757 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:02:06.757 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:02:06.757 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:02:06.757 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:02:06.757 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:02:06.757 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:02:06.757 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:02:06.758 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:02:06.758 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:02:06.758 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:02:07.480 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:02:07.481 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:02:07.900 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:02:08.043 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:02:08.044 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:02:08.044 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:02:08.044 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:02:08.044 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:02:08.701 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-23T00:02:09.482 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:02:11.115 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:02:11.367 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:02:11.367 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:02:11.367 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:02:11.367 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:02:12.183 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-23T00:02:12.559 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:02:12.582 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:02:12.605 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:02:12.628 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:02:12.656 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:02:12.686 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:02:12.718 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:02:12.747 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:02:12.776 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:02:12.803 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:02:12.829 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:02:12.831 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:02:12.831 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:02:12.831 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:02:12.831 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:02:12.832 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:02:12.832 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:02:12.832 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:02:12.832 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:02:12.832 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:02:12.832 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:02:12.832 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:02:12.833 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:02:12.833 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:02:13.597 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:02:13.597 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:02:13.916 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:02:14.389 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:02:15.140 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-23T00:02:15.598 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:02:17.115 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:02:17.661 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:02:18.466 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-23T00:02:18.823 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:02:18.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:02:18.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:02:18.893 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:02:18.919 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:02:18.946 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:02:18.972 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:02:18.999 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:02:19.030 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:02:19.071 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:02:19.098 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:02:19.099 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:02:19.099 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:02:19.100 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:02:19.100 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:02:19.100 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:02:19.100 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:02:19.100 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:02:19.100 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:02:19.100 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:02:19.101 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:02:19.101 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:02:19.101 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:02:19.101 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:02:19.995 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:02:20.038 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:02:20.039 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:02:20.603 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:02:20.603 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:02:20.603 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:02:20.603 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:02:21.221 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-23T00:02:22.040 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:02:23.154 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:02:23.900 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:02:23.900 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:02:23.901 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:02:23.901 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:02:23.901 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:02:24.612 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-23T00:02:24.854 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:02:24.879 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:02:24.902 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:02:24.926 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:02:24.964 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:02:24.999 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:02:25.032 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:02:25.065 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:02:25.095 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:02:25.149 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:02:25.189 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:02:25.191 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:02:25.191 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:02:25.191 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:02:25.192 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:02:25.192 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:02:25.192 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:02:25.192 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:02:25.192 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:02:25.192 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:02:25.192 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:02:25.193 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:02:25.193 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:02:25.193 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:02:25.749 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:02:25.749 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:02:26.035 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:02:26.726 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:02:27.325 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-23T00:02:27.750 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:02:29.851 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:02:30.452 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:02:30.452 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-23T00:02:30.729 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:02:30.751 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:02:30.774 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:02:30.797 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:02:30.823 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:02:30.854 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:02:30.884 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:02:30.913 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:02:30.940 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:02:30.986 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:02:31.025 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:02:31.027 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:02:31.027 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:02:31.027 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:02:31.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:02:31.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:02:31.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:02:31.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:02:31.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:02:31.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:02:31.028 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:02:31.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:02:31.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:02:31.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:02:31.639 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:02:31.639 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:02:32.569 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:02:33.094 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:02:33.095 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-23T00:02:33.640 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:02:35.424 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:02:36.031 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:02:36.032 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-23T00:02:36.358 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:02:36.381 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:02:36.405 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:02:36.430 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:02:36.462 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:02:36.493 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:02:36.523 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:02:36.550 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:02:36.579 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:02:36.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:02:36.632 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:02:36.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:02:36.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:02:36.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:02:36.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:02:36.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:02:36.634 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:02:36.635 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:02:36.635 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:02:36.635 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:02:36.635 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:02:36.635 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:02:36.635 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:02:36.635 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:02:37.350 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:02:37.350 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:02:38.179 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:02:38.721 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:02:38.722 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-23T00:02:39.120 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:02:39.351 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:02:41.310 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:02:41.768 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:02:41.969 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:02:41.996 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:02:42.006 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:02:42.006 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-23T00:02:42.020 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:02:42.045 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:02:42.072 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:02:42.104 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:02:42.132 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:02:42.158 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:02:42.195 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:02:42.221 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:02:42.279 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:02:42.281 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:02:42.281 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:02:42.281 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:02:42.281 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:02:42.281 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:02:42.282 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:02:42.282 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:02:42.282 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:02:42.282 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:02:42.282 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:02:42.282 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:02:42.282 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:02:42.283 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:02:42.990 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:02:42.990 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:02:44.467 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:02:44.990 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:02:44.991 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-23T00:02:44.992 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:02:45.361 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:02:47.660 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:02:47.685 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:02:47.710 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:02:47.734 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:02:47.760 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:02:47.786 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:02:47.818 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:02:47.848 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:02:47.876 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:02:47.901 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:02:47.916 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:02:47.962 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:02:47.964 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:02:47.965 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:02:47.965 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:02:47.965 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:02:47.965 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:02:47.965 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:02:47.965 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:02:47.965 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:02:47.966 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:02:47.966 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:02:47.966 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:02:47.966 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:02:47.966 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:02:48.212 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:02:48.525 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:02:48.526 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-23T00:02:48.694 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:02:48.694 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:02:50.695 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:02:50.900 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:02:51.151 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:02:51.572 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:02:51.573 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-23T00:02:53.384 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:02:54.006 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:02:54.033 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:02:54.241 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:02:54.264 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:02:54.288 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:02:54.314 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:02:54.340 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:02:54.390 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:02:54.427 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:02:54.461 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:02:54.493 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:02:54.524 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:02:54.526 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:02:54.526 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:02:54.527 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:02:54.527 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:02:54.527 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:02:54.527 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:02:54.527 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:02:54.527 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:02:54.527 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:02:54.528 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:02:54.528 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:02:54.528 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:02:54.528 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:02:54.577 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:02:54.578 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-23T00:02:55.228 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:02:55.229 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:02:57.135 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:02:57.160 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:02:57.230 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:02:57.791 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:02:57.791 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-23T00:03:00.062 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:03:00.146 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:03:00.271 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:03:00.298 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:03:00.316 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:03:00.339 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:03:00.366 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:03:00.392 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:03:00.418 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:03:00.448 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:03:00.472 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:03:00.499 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:03:00.525 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:03:00.527 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:03:00.527 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:03:00.528 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:03:00.528 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:03:00.528 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:03:00.528 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:03:00.528 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:03:00.528 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:03:00.529 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:03:00.529 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:03:00.529 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:03:00.529 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:03:00.529 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:03:00.668 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:03:00.669 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-23T00:03:01.239 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:03:01.240 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:03:03.241 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:03:03.373 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:03:03.900 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:03:03.901 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-23T00:03:04.136 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:03:06.252 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:03:06.448 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:03:06.858 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:03:06.958 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:03:06.959 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-23T00:03:07.103 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:03:07.129 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:03:07.153 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:03:07.184 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:03:07.210 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:03:07.236 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:03:07.263 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:03:07.290 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:03:07.317 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:03:07.345 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:03:07.346 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:03:07.346 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:03:07.347 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:03:07.347 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:03:07.347 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:03:07.347 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:03:07.347 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:03:07.347 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:03:07.347 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:03:07.348 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:03:07.348 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:03:07.348 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:03:07.348 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:03:08.009 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:03:08.009 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:03:09.529 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:03:10.010 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:03:10.041 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:03:10.042 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-23T00:03:10.144 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:03:12.653 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:03:12.674 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:03:12.696 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:03:12.702 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:03:12.724 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:03:12.751 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:03:12.779 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:03:12.807 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:03:12.833 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:03:12.860 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:03:12.892 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:03:12.899 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:03:12.926 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:03:12.928 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:03:12.928 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:03:12.928 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:03:12.928 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:03:12.928 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:03:12.929 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:03:12.929 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:03:12.929 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:03:12.929 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:03:12.929 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:03:12.929 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:03:12.929 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:03:12.930 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:03:13.680 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:03:13.681 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-23T00:03:13.866 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:03:13.867 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:03:15.868 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:03:16.026 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:03:16.153 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:03:16.687 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:03:16.688 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-23T00:03:18.948 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:03:18.971 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:03:18.973 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:03:18.988 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:03:18.996 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:03:19.021 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:03:19.048 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:03:19.074 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:03:19.100 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:03:19.126 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:03:19.154 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:03:19.181 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:03:19.208 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:03:19.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:03:19.209 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:03:19.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:03:19.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:03:19.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:03:19.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:03:19.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:03:19.210 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:03:19.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:03:19.211 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:03:19.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:03:19.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:03:19.211 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:03:19.511 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:03:19.512 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-23T00:03:19.979 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:03:19.980 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:03:21.980 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:03:21.981 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:03:22.170 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:03:22.630 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:03:22.630 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-23T00:03:24.803 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:03:24.828 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:03:24.847 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:03:24.851 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:03:24.874 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:03:24.900 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:03:24.926 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:03:24.953 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:03:24.979 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:03:25.005 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:03:25.031 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:03:25.038 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:03:25.062 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:03:25.065 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:03:25.065 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:03:25.065 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:03:25.065 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:03:25.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:03:25.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:03:25.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:03:25.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:03:25.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:03:25.066 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:03:25.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:03:25.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:03:25.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:03:25.376 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:03:25.376 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-23T00:03:25.835 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:03:25.835 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:03:27.796 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:03:27.837 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:03:28.207 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:03:28.353 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:03:28.353 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-23T00:03:30.692 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:03:30.717 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:03:30.741 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:03:30.769 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:03:30.795 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:03:30.823 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:03:30.849 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:03:30.875 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:03:30.901 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:03:30.902 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:03:30.911 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:03:30.934 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:03:30.963 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:03:30.965 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:03:30.966 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:03:30.966 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:03:30.966 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:03:30.966 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:03:30.966 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:03:30.966 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:03:30.966 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:03:30.967 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:03:30.967 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:03:30.967 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:03:30.967 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:03:30.967 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:03:31.598 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:03:31.599 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-23T00:03:31.867 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:03:31.867 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:03:33.868 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:03:34.033 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:03:34.033 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:03:34.033 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:03:34.033 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:03:34.256 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:03:34.701 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-23T00:03:36.694 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:03:36.729 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:03:36.760 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:03:36.792 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:03:36.820 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:03:36.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:03:36.874 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:03:36.900 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:03:36.927 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:03:36.955 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:03:36.987 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:03:36.988 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:03:36.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:03:36.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:03:36.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:03:36.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:03:36.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:03:36.989 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:03:36.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:03:36.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:03:36.990 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:03:36.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:03:36.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:03:36.990 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:03:37.330 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:03:37.446 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:03:37.474 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:03:37.474 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:03:37.925 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-23T00:03:39.476 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:03:40.169 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:03:41.024 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:03:41.614 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-23T00:03:42.236 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:03:42.271 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:03:42.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:03:42.320 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:03:42.353 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:03:42.388 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:03:42.419 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:03:42.445 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:03:42.472 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:03:42.499 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:03:42.542 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:03:42.543 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:03:42.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:03:42.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:03:42.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:03:42.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:03:42.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:03:42.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:03:42.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:03:42.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:03:42.545 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:03:42.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:03:42.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:03:42.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:03:43.161 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:03:43.244 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:03:43.244 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:03:43.814 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:03:43.815 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:03:43.815 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:03:43.815 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:03:44.439 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-23T00:03:45.245 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:03:46.543 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:03:47.196 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:03:47.197 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:03:47.197 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:03:47.197 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:03:47.197 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:03:47.987 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-23T00:03:48.372 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:03:48.399 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:03:48.425 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:03:48.449 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:03:48.476 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:03:48.502 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:03:48.530 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:03:48.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:03:48.584 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:03:48.610 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:03:48.637 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:03:48.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:03:48.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:03:48.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:03:48.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:03:48.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:03:48.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:03:48.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:03:48.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:03:48.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:03:48.640 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:03:48.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:03:48.641 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:03:48.641 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:03:49.464 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:03:49.466 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:03:49.466 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:03:50.206 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:03:50.723 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-23T00:03:51.467 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:03:52.174 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:03:53.802 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:03:54.328 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:03:54.332 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-23T00:03:54.351 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:03:54.377 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:03:54.400 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:03:54.426 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:03:54.454 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:03:54.479 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:03:54.506 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:03:54.532 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:03:54.560 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:03:54.589 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:03:54.590 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:03:54.590 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:03:54.591 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:03:54.591 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:03:54.591 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:03:54.591 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:03:54.591 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:03:54.591 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:03:54.591 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:03:54.592 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:03:54.592 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:03:54.592 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:03:54.592 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:03:54.940 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:03:55.356 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:03:55.356 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:03:56.709 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:03:56.709 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:03:56.709 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:03:56.709 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:03:56.709 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:03:57.346 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-23T00:03:57.357 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:03:57.827 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:03:59.944 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:03:59.944 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:03:59.944 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:03:59.945 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:03:59.945 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:04:00.236 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:04:00.265 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:04:00.288 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:04:00.312 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:04:00.338 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:04:00.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:04:00.394 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:04:00.420 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:04:00.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:04:00.475 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:04:00.501 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:04:00.504 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:04:00.504 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:04:00.504 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:04:00.504 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:04:00.504 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:04:00.504 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:04:00.505 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:04:00.505 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:04:00.505 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:04:00.505 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:04:00.505 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:04:00.505 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:04:00.506 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:04:00.582 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:04:00.595 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-23T00:04:00.970 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:04:00.970 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:04:02.971 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:04:03.389 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:04:04.060 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-23T00:04:04.198 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:04:05.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:04:05.643 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:04:05.660 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:04:05.683 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:04:05.732 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:04:05.770 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:04:05.804 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:04:05.834 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:04:05.864 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:04:05.895 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:04:05.923 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:04:05.925 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:04:05.925 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:04:05.925 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:04:05.925 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:04:05.925 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:04:05.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:04:05.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:04:05.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:04:05.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:04:05.926 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:04:05.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:04:05.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:04:05.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:04:06.530 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:04:06.530 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:04:06.800 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:04:06.800 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:04:06.800 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:04:06.800 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:04:06.800 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:04:06.948 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:04:07.295 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-23T00:04:08.531 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:04:09.757 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:04:09.757 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:04:09.758 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:04:09.758 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:04:09.758 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:04:10.186 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:04:10.698 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-23T00:04:11.435 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:04:11.473 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:04:11.500 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:04:11.524 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:04:11.550 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:04:11.577 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:04:11.608 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:04:11.635 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:04:11.662 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:04:11.690 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:04:11.716 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:04:11.717 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:04:11.717 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:04:11.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:04:11.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:04:11.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:04:11.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:04:11.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:04:11.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:04:11.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:04:11.719 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:04:11.719 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:04:11.719 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:04:11.719 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:04:12.426 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:04:12.426 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:04:12.951 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:04:12.989 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:04:13.589 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:04:13.589 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-23T00:04:14.427 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:04:16.184 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:04:16.464 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:04:17.266 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:04:17.266 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-23T00:04:17.625 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:04:17.650 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:04:17.674 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:04:17.698 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:04:17.724 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:04:17.750 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:04:17.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:04:17.807 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:04:17.835 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:04:17.867 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:04:17.896 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:04:17.898 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:04:17.898 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:04:17.898 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:04:17.899 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:04:17.899 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:04:17.899 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:04:17.899 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:04:17.899 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:04:17.899 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:04:17.899 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:04:17.899 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:04:17.900 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:04:17.900 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:04:18.639 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:04:18.639 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:04:18.771 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:04:19.426 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:04:19.980 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:04:19.980 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-23T00:04:20.640 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:04:22.196 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:04:22.475 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:04:23.226 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:04:23.227 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-23T00:04:24.959 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:04:25.260 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:04:26.070 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:04:26.071 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-23T00:04:27.952 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:04:28.529 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:04:28.961 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:04:28.962 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-23T00:04:30.498 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:04:31.029 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:04:31.611 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:04:31.611 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-23T00:04:33.218 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:04:33.629 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:04:33.669 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:04:33.704 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:04:33.732 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:04:33.759 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:04:33.800 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:04:33.812 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:04:33.847 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:04:33.885 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:04:33.921 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:04:33.956 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:04:34.000 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:04:34.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:04:34.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:04:34.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:04:34.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:04:34.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:04:34.003 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:04:34.003 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:04:34.003 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:04:34.003 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:04:34.003 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:04:34.003 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:04:34.004 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:04:34.004 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:04:34.276 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:04:34.276 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-23T00:04:34.671 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:04:34.672 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:04:35.832 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:04:36.203 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:04:36.673 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:04:36.818 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:04:36.819 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-23T00:04:39.306 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:04:39.328 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:04:39.351 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:04:39.374 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:04:39.401 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:04:39.426 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:04:39.452 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:04:39.478 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:04:39.505 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:04:39.532 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:04:39.559 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:04:39.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:04:39.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:04:39.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:04:39.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:04:39.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:04:39.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:04:39.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:04:39.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:04:39.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:04:39.562 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:04:39.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:04:39.562 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:04:39.563 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:04:39.791 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:04:40.114 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:04:40.115 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:04:40.203 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:04:40.452 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:04:40.453 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-23T00:04:42.116 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:04:42.444 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:04:42.855 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:04:43.195 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:04:43.195 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-23T00:04:45.117 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:04:45.152 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:04:45.180 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:04:45.220 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:04:45.258 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:04:45.293 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:04:45.328 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:04:45.382 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:04:45.424 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:04:45.461 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:04:45.494 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:04:45.496 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:04:45.496 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:04:45.496 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:04:45.496 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:04:45.496 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:04:45.496 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:04:45.497 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:04:45.497 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:04:45.497 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:04:45.497 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:04:45.497 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:04:45.497 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:04:45.497 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:04:46.040 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:04:46.169 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:04:46.169 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:04:46.375 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:04:46.539 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:04:46.540 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-23T00:04:48.170 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:04:48.497 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:04:48.904 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:04:49.299 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:04:49.299 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-23T00:04:51.407 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:04:51.437 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:04:51.475 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:04:51.506 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:04:51.535 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:04:51.564 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:04:51.591 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:04:51.626 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:04:51.659 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:04:51.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:04:51.721 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:04:51.731 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:04:51.732 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:04:51.732 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:04:51.733 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:04:51.733 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:04:51.733 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:04:51.733 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:04:51.733 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:04:51.734 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:04:51.734 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:04:51.734 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:04:51.734 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:04:51.735 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:04:52.212 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:04:52.257 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:04:52.355 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:04:52.356 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:04:53.020 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:04:53.021 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-23T00:04:54.357 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:04:54.827 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:04:54.894 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:04:55.962 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:04:55.963 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-23T00:04:57.384 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:04:57.424 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:04:57.460 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:04:57.491 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:04:57.522 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:04:57.550 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:04:57.578 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:04:57.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:04:57.634 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:04:57.678 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:04:57.716 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:04:57.717 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:04:57.717 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:04:57.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:04:57.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:04:57.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:04:57.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:04:57.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:04:57.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:04:57.718 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:04:57.718 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:04:57.719 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:04:57.719 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:04:57.719 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:04:58.362 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:04:58.362 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:04:58.404 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:04:58.909 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:04:59.352 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:04:59.353 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-23T00:05:00.363 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:05:00.967 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:05:01.610 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:05:02.380 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:05:02.381 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-23T00:05:03.251 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:05:04.102 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:05:04.169 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:05:04.214 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:05:04.255 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:05:04.283 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:05:04.311 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:05:04.338 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:05:04.366 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:05:04.393 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:05:04.420 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:05:04.422 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:05:04.422 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:05:04.422 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:05:04.422 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:05:04.423 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:05:04.423 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:05:04.423 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:05:04.423 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:05:04.423 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:05:04.423 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:05:04.423 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:05:04.424 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:05:04.424 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:05:04.433 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:05:05.046 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:05:05.063 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:05:05.063 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:05:05.601 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:05:05.602 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-23T00:05:07.065 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:05:07.288 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:05:07.548 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:05:08.290 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:05:08.291 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-23T00:05:10.366 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:05:10.390 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:05:10.414 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:05:10.436 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:05:10.462 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:05:10.495 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:05:10.514 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:05:10.543 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:05:10.572 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:05:10.597 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:05:10.624 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:05:10.625 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:05:10.625 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:05:10.626 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:05:10.626 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:05:10.626 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:05:10.626 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:05:10.626 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:05:10.626 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:05:10.626 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:05:10.627 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:05:10.627 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:05:10.627 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:05:10.627 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:05:10.738 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:05:11.227 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:05:11.290 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:05:11.291 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:05:11.481 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:05:11.482 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-23T00:05:13.292 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:05:13.420 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:05:13.811 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:05:13.921 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:05:13.921 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-23T00:05:16.780 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:05:16.807 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:05:16.832 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:05:16.856 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:05:16.882 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:05:16.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:05:16.955 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:05:17.005 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:05:17.039 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:05:17.073 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:05:17.087 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:05:17.087 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:05:17.087 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:05:17.087 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:05:17.116 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:05:17.118 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:05:17.118 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:05:17.118 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:05:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:05:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:05:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:05:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:05:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:05:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:05:17.119 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:05:17.120 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:05:17.120 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:05:17.120 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:05:17.218 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:05:17.744 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-23T00:05:18.061 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:05:18.061 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:05:19.887 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:05:19.888 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:05:19.888 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:05:19.888 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:05:19.888 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:05:20.062 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:05:20.243 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:05:20.407 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-23T00:05:23.159 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:05:23.685 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:05:24.025 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:05:24.072 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:05:24.113 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:05:24.153 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:05:24.194 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:05:24.212 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-23T00:05:24.229 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:05:24.244 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:05:24.260 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:05:24.288 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:05:24.326 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:05:24.358 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:05:24.360 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:05:24.360 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:05:24.360 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:05:24.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:05:24.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:05:24.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:05:24.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:05:24.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:05:24.361 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:05:24.361 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:05:24.362 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:05:24.362 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:05:24.362 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:05:25.227 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:05:25.227 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:05:26.664 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:05:26.976 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:05:27.142 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-23T00:05:27.228 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:05:30.232 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:05:30.239 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:05:30.338 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:05:30.361 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:05:30.384 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:05:30.406 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:05:30.433 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:05:30.459 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:05:30.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:05:30.511 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:05:30.537 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:05:30.566 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:05:30.590 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:05:30.592 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:05:30.592 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:05:30.593 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:05:30.593 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:05:30.593 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:05:30.593 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:05:30.593 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:05:30.593 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:05:30.593 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:05:30.593 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:05:30.594 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:05:30.594 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:05:30.594 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:05:31.235 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-23T00:05:31.395 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:05:31.395 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:05:32.953 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:05:33.085 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:05:33.085 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:05:33.085 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:05:33.086 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:05:33.086 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:05:33.396 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:05:33.776 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-23T00:05:36.330 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:05:36.354 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:05:36.376 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:05:36.401 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:05:36.428 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:05:36.461 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:05:36.491 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:05:36.518 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:05:36.544 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:05:36.571 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:05:36.610 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:05:36.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:05:36.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:05:36.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:05:36.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:05:36.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:05:36.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:05:36.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:05:36.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:05:36.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:05:36.614 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:05:36.614 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:05:36.614 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:05:36.614 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:05:36.616 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:05:37.203 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:05:37.211 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:05:37.212 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:05:37.397 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-23T00:05:39.212 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:05:39.390 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:05:39.732 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:05:40.017 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-23T00:05:42.395 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:05:42.418 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:05:42.442 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:05:42.465 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:05:42.492 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:05:42.518 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:05:42.546 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:05:42.573 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:05:42.599 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:05:42.627 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:05:42.655 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:05:42.655 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:05:42.656 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:05:42.656 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:05:42.656 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:05:42.656 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:05:42.656 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:05:42.657 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:05:42.657 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:05:42.657 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:05:42.657 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:05:42.657 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:05:42.657 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:05:42.657 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:05:42.664 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:05:42.664 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:05:42.665 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:05:42.665 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:05:42.665 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:05:43.252 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:05:43.252 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:05:43.414 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-23T00:05:45.253 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:05:45.354 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:05:45.354 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:05:45.355 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:05:45.355 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:05:45.866 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-23T00:05:47.754 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:05:47.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:05:47.802 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:05:47.826 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:05:47.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:05:47.879 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:05:47.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:05:47.937 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:05:47.958 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:05:47.985 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:05:47.994 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:05:48.012 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:05:48.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:05:48.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:05:48.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:05:48.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:05:48.014 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:05:48.015 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:05:48.015 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:05:48.015 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:05:48.015 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:05:48.015 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:05:48.016 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:05:48.016 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:05:48.016 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:05:48.652 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:05:48.653 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:05:48.795 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-23T00:05:50.654 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:05:50.750 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:05:51.273 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-23T00:05:53.243 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:05:53.703 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:05:54.158 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:05:54.158 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-23T00:05:55.755 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:05:56.130 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:05:56.911 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:05:56.911 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-23T00:05:58.982 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:05:59.203 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:05:59.568 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:05:59.569 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-23T00:06:01.609 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:06:01.917 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:06:02.167 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:06:02.167 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-23T00:06:03.304 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:06:03.348 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:06:03.379 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:06:03.411 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:06:03.442 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:06:03.469 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:06:03.497 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:06:03.522 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:06:03.551 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:06:03.577 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:06:03.602 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:06:03.604 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:06:03.605 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:06:03.605 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:06:03.605 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:06:03.605 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:06:03.605 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:06:03.605 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:06:03.605 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:06:03.606 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:06:03.606 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:06:03.606 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:06:03.606 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:06:03.606 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:06:04.368 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:06:04.368 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:06:04.743 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:06:05.256 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:06:05.257 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-23T00:06:05.309 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:06:06.369 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:06:07.755 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:06:07.800 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:06:08.513 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:06:08.513 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-23T00:06:08.717 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:06:09.573 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:06:09.621 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:06:09.674 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:06:09.709 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:06:09.743 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:06:09.772 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:06:09.800 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:06:09.827 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:06:09.855 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:06:09.882 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:06:09.884 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:06:09.884 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:06:09.885 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:06:09.885 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:06:09.885 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:06:09.885 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:06:09.885 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:06:09.885 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:06:09.886 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:06:09.886 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:06:09.886 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:06:09.886 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:06:09.886 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:06:10.595 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:06:10.595 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:06:11.145 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:06:11.259 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:06:11.847 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:06:11.848 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-23T00:06:12.596 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:06:13.905 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:06:14.020 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:06:14.821 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:06:14.821 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-23T00:06:15.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:06:16.664 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:06:16.705 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:06:16.739 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:06:16.771 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:06:16.800 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:06:16.827 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:06:16.861 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:06:16.893 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:06:16.927 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:06:16.949 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:06:16.950 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:06:16.950 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:06:16.950 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:06:16.951 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:06:16.951 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:06:16.951 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:06:16.951 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:06:16.951 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:06:16.951 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:06:16.951 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:06:16.952 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:06:16.952 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:06:16.952 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:06:17.263 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:06:17.398 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:06:17.488 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:06:17.489 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:06:18.180 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:06:18.181 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-23T00:06:19.490 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:06:20.015 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:06:20.108 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:06:20.678 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:06:20.679 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-23T00:06:22.202 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:06:22.227 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:06:22.256 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:06:22.290 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:06:22.342 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:06:22.370 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:06:22.401 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:06:22.425 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:06:22.477 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:06:22.522 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:06:22.566 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:06:22.568 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:06:22.568 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:06:22.569 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:06:22.569 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:06:22.569 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:06:22.569 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:06:22.569 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:06:22.569 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:06:22.570 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:06:22.570 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:06:22.570 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:06:22.570 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:06:22.570 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:06:23.452 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:06:23.453 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:06:23.770 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:06:23.997 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:06:24.722 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:06:24.722 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-23T00:06:25.453 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:06:26.556 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:06:26.787 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:06:27.651 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:06:27.651 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-23T00:06:28.440 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:06:28.440 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:06:28.461 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:06:28.484 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:06:28.511 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:06:28.537 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:06:28.563 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:06:28.589 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:06:28.616 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:06:28.643 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:06:28.671 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:06:28.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:06:28.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:06:28.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:06:28.673 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:06:28.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:06:28.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:06:28.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:06:28.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:06:28.674 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:06:28.675 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:06:28.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:06:28.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:06:28.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:06:29.374 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:06:29.375 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:06:30.277 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:06:30.333 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:06:31.067 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:06:31.068 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-23T00:06:31.375 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:06:32.914 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:06:33.403 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:06:33.699 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:06:34.123 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:06:34.123 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-23T00:06:34.554 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:06:34.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:06:34.646 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:06:34.683 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:06:34.720 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:06:34.750 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:06:34.781 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:06:34.808 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:06:34.837 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:06:34.864 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:06:34.865 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:06:34.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:06:34.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:06:34.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:06:34.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:06:34.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:06:34.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:06:34.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:06:34.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:06:34.867 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:06:34.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:06:34.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:06:34.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:06:35.652 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:06:35.652 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:06:35.701 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:06:36.668 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:06:37.181 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:06:37.182 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-23T00:06:37.654 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:06:38.296 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:06:40.107 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:06:40.803 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:06:40.839 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:06:40.846 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:06:40.846 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-23T00:06:40.875 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:06:40.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:06:40.940 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:06:40.968 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:06:41.001 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:06:41.035 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:06:41.062 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:06:41.096 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:06:41.123 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:06:41.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:06:41.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:06:41.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:06:41.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:06:41.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:06:41.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:06:41.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:06:41.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:06:41.125 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:06:41.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:06:41.126 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:06:41.126 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:06:41.762 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:06:41.763 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:06:42.301 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:06:43.271 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:06:43.764 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:06:43.788 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:06:43.789 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-23T00:06:44.875 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:06:46.678 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:06:46.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:06:46.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:06:46.711 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:06:46.736 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:06:46.764 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:06:46.790 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:06:46.816 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:06:46.843 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:06:46.870 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:06:46.897 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:06:46.923 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:06:46.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:06:46.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:06:46.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:06:46.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:06:46.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:06:46.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:06:46.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:06:46.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:06:46.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:06:46.927 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:06:46.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:06:46.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:06:46.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:06:47.327 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:06:47.327 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-23T00:06:47.799 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:06:47.800 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:06:48.012 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:06:49.528 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:06:49.801 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:06:50.067 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:06:50.067 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-23T00:06:50.564 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:06:52.295 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:06:52.318 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:06:52.342 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:06:52.365 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:06:52.391 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:06:52.421 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:06:52.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:06:52.473 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:06:52.499 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:06:52.526 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:06:52.551 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:06:52.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:06:52.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:06:52.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:06:52.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:06:52.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:06:52.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:06:52.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:06:52.555 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:06:52.556 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:06:52.556 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:06:52.556 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:06:52.556 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:06:52.557 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:06:53.171 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:06:53.247 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:06:53.253 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:06:53.799 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:06:53.800 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-23T00:06:55.254 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:06:55.284 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:06:55.986 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:06:56.591 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:06:56.591 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-23T00:06:57.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:06:57.827 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:06:57.850 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:06:57.873 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:06:57.900 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:06:57.927 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:06:57.953 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:06:57.979 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:06:57.987 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:06:58.007 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:06:58.034 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:06:58.060 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:06:58.062 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:06:58.062 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:06:58.062 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:06:58.063 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:06:58.063 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:06:58.063 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:06:58.063 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:06:58.063 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:06:58.063 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:06:58.064 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:06:58.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:06:58.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:06:58.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:06:58.689 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:06:58.690 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:06:59.009 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:06:59.521 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-23T00:07:00.690 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:07:01.299 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:07:02.126 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:07:02.648 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-23T00:07:03.120 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:07:03.143 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:07:03.167 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:07:03.191 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:07:03.218 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:07:03.245 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:07:03.270 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:07:03.300 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:07:03.325 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:07:03.352 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:07:03.378 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:07:03.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:07:03.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:07:03.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:07:03.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:07:03.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:07:03.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:07:03.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:07:03.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:07:03.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:07:03.383 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:07:03.383 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:07:03.383 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:07:03.383 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:07:04.118 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:07:04.139 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:07:04.140 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:07:04.932 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:07:04.933 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:07:04.933 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:07:04.933 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:07:04.933 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:07:05.467 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-23T00:07:06.141 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:07:07.301 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:07:07.902 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:07:07.902 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:07:07.902 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:07:07.902 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:07:08.649 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-23T00:07:09.011 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:07:09.034 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:07:09.058 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:07:09.081 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:07:09.107 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:07:09.134 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:07:09.161 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:07:09.188 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:07:09.214 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:07:09.242 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:07:09.269 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:07:09.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:07:09.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:07:09.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:07:09.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:07:09.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:07:09.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:07:09.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:07:09.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:07:09.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:07:09.272 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:07:09.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:07:09.273 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:07:09.273 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:07:09.970 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:07:09.970 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:07:10.041 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:07:10.886 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:07:11.720 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-23T00:07:11.971 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:07:13.326 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:07:14.380 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:07:14.600 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:07:14.627 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:07:14.653 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:07:14.677 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:07:14.703 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:07:14.730 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:07:14.756 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:07:14.782 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:07:14.809 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:07:14.835 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:07:14.861 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:07:14.862 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:07:14.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:07:14.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:07:14.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:07:14.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:07:14.863 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:07:14.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:07:14.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:07:14.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:07:14.864 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:07:14.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:07:14.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:07:14.864 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:07:14.943 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-23T00:07:15.281 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:07:15.281 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:07:16.155 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:07:16.852 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:07:16.852 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:07:16.852 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:07:16.852 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:07:17.282 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:07:17.415 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-23T00:07:19.266 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:07:20.327 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:07:20.327 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:07:20.327 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:07:20.327 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:07:20.327 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:07:20.848 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-23T00:07:22.082 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:07:22.849 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:07:22.849 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:07:22.849 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:07:22.850 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:07:22.850 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:07:23.404 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-23T00:07:25.300 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:07:25.392 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:07:25.392 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:07:25.392 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:07:25.392 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:07:26.124 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-23T00:07:27.961 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:07:28.143 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:07:28.144 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:07:28.144 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:07:28.144 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:07:28.144 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:07:28.789 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-23T00:07:30.532 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:07:30.571 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:07:30.603 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:07:30.630 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:07:30.659 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:07:30.686 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:07:30.712 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:07:30.738 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:07:30.765 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:07:30.792 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:07:30.818 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:07:30.819 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:07:30.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:07:30.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:07:30.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:07:30.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:07:30.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:07:30.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:07:30.820 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:07:30.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:07:30.821 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:07:30.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:07:30.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:07:30.821 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:07:31.303 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:07:31.342 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:07:31.375 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:07:31.376 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:07:32.004 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-23T00:07:33.377 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:07:33.844 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:07:33.909 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:07:34.502 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:07:34.502 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-23T00:07:36.226 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:07:37.072 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:07:37.077 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:07:37.107 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:07:37.134 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:07:37.162 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:07:37.188 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:07:37.215 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:07:37.242 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:07:37.270 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:07:37.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:07:37.323 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:07:37.324 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:07:37.324 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:07:37.325 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:07:37.325 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:07:37.325 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:07:37.325 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:07:37.325 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:07:37.326 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:07:37.326 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:07:37.326 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:07:37.326 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:07:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:07:37.327 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:07:37.672 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:07:37.673 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-23T00:07:38.013 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:07:38.014 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:07:39.701 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:07:40.015 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:07:40.229 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:07:40.230 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-23T00:07:42.391 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:07:42.418 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:07:42.449 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:07:42.478 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:07:42.529 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:07:42.565 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:07:42.594 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:07:42.595 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:07:42.623 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:07:42.649 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:07:42.687 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:07:42.722 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:07:42.724 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:07:42.724 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:07:42.725 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:07:42.725 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:07:42.725 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:07:42.725 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:07:42.725 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:07:42.725 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:07:42.725 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:07:42.726 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:07:42.726 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:07:42.726 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:07:42.726 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:07:43.095 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:07:43.096 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-23T00:07:43.442 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:07:43.443 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:07:45.076 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:07:45.443 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:07:45.582 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:07:45.582 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-23T00:07:47.711 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:07:47.954 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:07:48.236 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:07:48.478 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:07:48.478 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-23T00:07:48.562 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:07:48.588 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:07:48.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:07:48.642 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:07:48.668 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:07:48.694 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:07:48.721 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:07:48.748 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:07:48.775 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:07:48.801 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:07:48.803 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:07:48.803 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:07:48.803 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:07:48.803 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:07:48.803 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:07:48.803 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:07:48.804 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:07:48.804 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:07:48.804 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:07:48.804 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:07:48.804 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:07:48.804 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:07:48.804 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:07:49.426 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:07:49.427 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:07:50.727 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:07:50.834 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:07:51.235 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:07:51.236 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-23T00:07:51.428 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:07:54.303 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:07:54.324 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:07:54.394 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:07:54.424 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:07:54.446 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:07:54.469 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:07:54.496 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:07:54.523 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:07:54.549 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:07:54.576 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:07:54.602 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:07:54.631 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:07:54.674 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:07:54.677 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:07:54.677 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:07:54.677 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:07:54.677 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:07:54.678 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:07:54.678 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:07:54.678 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:07:54.678 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:07:54.678 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:07:54.679 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:07:54.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:07:54.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:07:54.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:07:55.009 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:07:55.010 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-23T00:07:55.475 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:07:55.475 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:07:57.079 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:07:57.276 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:07:57.476 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:07:57.619 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:07:57.620 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-23T00:08:00.288 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:08:00.317 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:08:00.337 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:08:00.345 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:08:00.376 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:08:00.405 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:08:00.436 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:08:00.464 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:08:00.491 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:08:00.514 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:08:00.517 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:08:00.545 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:08:00.571 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:08:00.572 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:08:00.572 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:08:00.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:08:00.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:08:00.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:08:00.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:08:00.573 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:08:00.574 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:08:00.574 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:08:00.574 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:08:00.574 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:08:00.575 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:08:00.575 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:08:01.376 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:08:01.377 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-23T00:08:01.529 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:08:01.529 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:08:03.136 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:08:03.310 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:08:03.530 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:08:03.984 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:08:03.984 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-23T00:08:06.652 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:08:06.967 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:08:07.342 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:08:07.502 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:08:07.541 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:08:07.542 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-23T00:08:07.545 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:08:07.581 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:08:07.612 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:08:07.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:08:07.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:08:07.695 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:08:07.720 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:08:07.757 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:08:07.790 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:08:07.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:08:07.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:08:07.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:08:07.791 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:08:07.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:08:07.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:08:07.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:08:07.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:08:07.792 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:08:07.792 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:08:07.793 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:08:07.793 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:08:07.793 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:08:08.427 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:08:08.427 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:08:09.844 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:08:09.921 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:08:10.372 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:08:10.373 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-23T00:08:10.428 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:08:13.172 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:08:13.330 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:08:13.804 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:08:13.804 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:08:13.825 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:08:13.849 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:08:13.879 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:08:13.906 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:08:13.935 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:08:13.938 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:08:13.939 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-23T00:08:13.963 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:08:13.989 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:08:14.016 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:08:14.050 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:08:14.051 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:08:14.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:08:14.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:08:14.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:08:14.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:08:14.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:08:14.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:08:14.053 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:08:14.053 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:08:14.053 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:08:14.053 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:08:14.053 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:08:14.053 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:08:14.626 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:08:14.626 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:08:16.045 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:08:16.140 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:08:16.570 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:08:16.571 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-23T00:08:16.627 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:08:19.336 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:08:19.371 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:08:19.403 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:08:19.431 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:08:19.458 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:08:19.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:08:19.517 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:08:19.540 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:08:19.565 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:08:19.591 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:08:19.618 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:08:19.644 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:08:19.646 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:08:19.646 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:08:19.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:08:19.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:08:19.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:08:19.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:08:19.647 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:08:19.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:08:19.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:08:19.648 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:08:19.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:08:19.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:08:19.648 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:08:19.763 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:08:20.350 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:08:20.351 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:08:20.518 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:08:20.518 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-23T00:08:21.912 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:08:22.352 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:08:22.440 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:08:23.020 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:08:23.020 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-23T00:08:25.355 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:08:25.693 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:08:25.718 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:08:25.742 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:08:25.767 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:08:25.794 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:08:25.820 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:08:25.853 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:08:25.882 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:08:25.913 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:08:25.948 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:08:25.999 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:08:26.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:08:26.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:08:26.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:08:26.001 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:08:26.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:08:26.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:08:26.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:08:26.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:08:26.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:08:26.002 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:08:26.002 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:08:26.003 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:08:26.003 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:08:26.280 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:08:26.641 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:08:26.641 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:08:26.950 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:08:26.951 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-23T00:08:28.068 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:08:28.642 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:08:28.805 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:08:29.483 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:08:29.483 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-23T00:08:31.452 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:08:31.471 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:08:31.498 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:08:31.526 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:08:31.556 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:08:31.580 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:08:31.617 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:08:31.637 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:08:31.660 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:08:31.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:08:31.713 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:08:31.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:08:31.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:08:31.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:08:31.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:08:31.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:08:31.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:08:31.715 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:08:31.716 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:08:31.716 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:08:31.716 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:08:31.716 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:08:31.716 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:08:32.319 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:08:32.320 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:08:32.346 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:08:32.352 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:08:32.970 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:08:32.971 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-23T00:08:34.320 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:08:34.955 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:08:35.156 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:08:35.538 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:08:35.538 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-23T00:08:37.053 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:08:37.087 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:08:37.144 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:08:37.217 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:08:37.245 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:08:37.280 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:08:37.311 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:08:37.343 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:08:37.373 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:08:37.403 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:08:37.429 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:08:37.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:08:37.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:08:37.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:08:37.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:08:37.431 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:08:37.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:08:37.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:08:37.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:08:37.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:08:37.432 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:08:37.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:08:37.432 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:08:37.433 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:08:38.078 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:08:38.078 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:08:38.737 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:08:38.737 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:08:38.738 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:08:38.738 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:08:39.291 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-23T00:08:40.079 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:08:41.456 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:08:41.457 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:08:41.457 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:08:41.457 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:08:41.457 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:08:42.150 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-23T00:08:42.745 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:08:42.777 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:08:42.806 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:08:42.830 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:08:42.873 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:08:42.899 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:08:42.925 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:08:42.952 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:08:42.983 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:08:43.013 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:08:43.041 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:08:43.042 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:08:43.042 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:08:43.042 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:08:43.042 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:08:43.043 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:08:43.043 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:08:43.043 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:08:43.043 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:08:43.043 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:08:43.044 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:08:43.044 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:08:43.044 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:08:43.044 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:08:43.805 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:08:43.805 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:08:44.378 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:08:44.891 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-23T00:08:45.806 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:08:47.143 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:08:47.799 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-23T00:08:48.449 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:08:49.318 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:08:49.351 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:08:49.377 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:08:49.418 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:08:49.454 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:08:49.484 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:08:49.512 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:08:49.540 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:08:49.567 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:08:49.597 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:08:49.624 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:08:49.625 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:08:49.625 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:08:49.625 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:08:49.627 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:08:49.627 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:08:49.627 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:08:49.628 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:08:49.628 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:08:49.628 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:08:49.628 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:08:49.628 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:08:49.628 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:08:49.628 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:08:50.067 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:08:50.067 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:08:50.067 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:08:50.067 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:08:50.068 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:08:50.382 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:08:50.382 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:08:50.685 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-23T00:08:51.994 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:08:52.383 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:08:52.614 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:08:52.614 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:08:52.614 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:08:52.615 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:08:52.615 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:08:53.308 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-23T00:08:55.427 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:08:55.476 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:08:55.500 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:08:55.524 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:08:55.548 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:08:55.583 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:08:55.603 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:08:55.629 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:08:55.656 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:08:55.682 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:08:55.709 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:08:55.735 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:08:55.738 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:08:55.738 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:08:55.738 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:08:55.738 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:08:55.739 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:08:55.739 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:08:55.739 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:08:55.739 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:08:55.739 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:08:55.739 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:08:55.739 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:08:55.740 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:08:55.740 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:08:56.483 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:08:56.484 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:08:56.554 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:08:57.030 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-23T00:08:58.065 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:08:58.485 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:08:59.284 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:09:00.116 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-23T00:09:01.372 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:09:01.721 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:09:01.755 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:09:01.784 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:09:01.812 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:09:01.840 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:09:01.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:09:01.907 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:09:01.938 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:09:01.968 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:09:01.998 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:09:02.050 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:09:02.051 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:09:02.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:09:02.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:09:02.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:09:02.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:09:02.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:09:02.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:09:02.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:09:02.053 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:09:02.053 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:09:02.053 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:09:02.053 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:09:02.053 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:09:02.615 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:09:02.616 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:09:02.897 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:09:02.897 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:09:02.897 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:09:02.897 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:09:02.898 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:09:03.372 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-23T00:09:04.190 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:09:04.617 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:09:05.903 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:09:05.904 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:09:05.904 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:09:05.904 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:09:06.584 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-23T00:09:07.593 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:09:07.613 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:09:07.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:09:07.660 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:09:07.670 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:09:07.696 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:09:07.722 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:09:07.748 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:09:07.777 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:09:07.802 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:09:07.828 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:09:07.854 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:09:07.856 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:09:07.856 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:09:07.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:09:07.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:09:07.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:09:07.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:09:07.857 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:09:07.858 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:09:07.858 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:09:07.858 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:09:07.858 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:09:07.858 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:09:07.858 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:09:08.869 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:09:08.869 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:09:09.334 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:09:09.855 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-23T00:09:10.599 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:09:10.870 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:09:12.601 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:09:13.415 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-23T00:09:13.786 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:09:14.390 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:09:14.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:09:14.694 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:09:14.730 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:09:14.761 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:09:14.789 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:09:14.815 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:09:14.842 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:09:14.869 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:09:14.896 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:09:14.925 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:09:14.925 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:09:14.925 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:09:14.925 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:09:14.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:09:14.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:09:14.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:09:14.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:09:14.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:09:14.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:09:14.926 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:09:14.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:09:14.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:09:14.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:09:15.698 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:09:15.699 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:09:16.052 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:09:16.608 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:09:16.609 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-23T00:09:17.212 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:09:17.700 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:09:19.463 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:09:20.329 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:09:20.329 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-23T00:09:20.375 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:09:20.658 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:09:20.682 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:09:20.705 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:09:20.730 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:09:20.781 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:09:20.824 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:09:20.875 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:09:20.917 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:09:20.956 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:09:20.988 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:09:21.040 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:09:21.042 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:09:21.042 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:09:21.043 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:09:21.043 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:09:21.043 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:09:21.043 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:09:21.043 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:09:21.043 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:09:21.043 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:09:21.044 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:09:21.044 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:09:21.044 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:09:21.044 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:09:21.711 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:09:21.711 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:09:22.711 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:09:23.199 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:09:23.199 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-23T00:09:23.210 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:09:23.712 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:09:26.228 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:09:26.387 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:09:26.752 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:09:26.796 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:09:26.806 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:09:26.806 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-23T00:09:26.831 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:09:26.860 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:09:26.893 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:09:26.916 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:09:26.943 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:09:26.968 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:09:26.995 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:09:27.022 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:09:27.049 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:09:27.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:09:27.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:09:27.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:09:27.050 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:09:27.051 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:09:27.051 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:09:27.051 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:09:27.051 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:09:27.051 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:09:27.051 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:09:27.051 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:09:27.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:09:27.052 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:09:27.747 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:09:27.748 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:09:29.040 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:09:29.207 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:09:29.571 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:09:29.571 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-23T00:09:29.749 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:09:32.416 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:09:32.522 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:09:32.665 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:09:32.689 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:09:32.722 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:09:32.749 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:09:32.779 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:09:32.811 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:09:32.841 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:09:32.870 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:09:32.894 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:09:32.925 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:09:32.947 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:09:32.952 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:09:32.952 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:09:32.952 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:09:32.952 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:09:32.952 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:09:32.952 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:09:32.953 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:09:32.953 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:09:32.953 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:09:32.953 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:09:32.953 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:09:32.953 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:09:32.953 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:09:33.449 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:09:33.449 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-23T00:09:33.601 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:09:33.601 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:09:35.159 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:09:35.393 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:09:35.602 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:09:36.125 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:09:36.126 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-23T00:09:38.490 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:09:38.514 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:09:38.538 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:09:38.573 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:09:38.609 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:09:38.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:09:38.669 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:09:38.693 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:09:38.731 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:09:38.732 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:09:38.765 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:09:38.795 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:09:38.797 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:09:38.797 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:09:38.797 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:09:38.797 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:09:38.797 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:09:38.798 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:09:38.798 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:09:38.798 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:09:38.798 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:09:38.798 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:09:38.798 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:09:38.798 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:09:38.799 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:09:39.253 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:09:39.361 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:09:39.362 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-23T00:09:39.526 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:09:39.527 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:09:41.522 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:09:41.527 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:09:41.873 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:09:42.360 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:09:42.361 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-23T00:09:44.343 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:09:44.343 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:09:44.343 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:09:44.343 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:09:44.344 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:09:44.344 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:09:44.355 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:09:44.381 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:09:44.426 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:09:44.471 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:09:44.505 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:09:44.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:09:44.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:09:44.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:09:44.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:09:44.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:09:44.508 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:09:44.509 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:09:44.509 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:09:44.509 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:09:44.509 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:09:44.509 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:09:44.509 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:09:44.509 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:09:45.338 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:09:45.338 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:09:45.419 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:09:45.425 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:09:46.001 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:09:46.001 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-23T00:09:47.339 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:09:47.837 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:09:48.163 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:09:48.374 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:09:48.374 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-23T00:09:50.264 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:09:50.297 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:09:50.327 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:09:50.354 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:09:50.382 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:09:50.409 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:09:50.438 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:09:50.462 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:09:50.489 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:09:50.516 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:09:50.542 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:09:50.543 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:09:50.543 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:09:50.543 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:09:50.543 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:09:50.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:09:50.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:09:50.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:09:50.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:09:50.544 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:09:50.544 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:09:50.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:09:50.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:09:50.545 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:09:51.172 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:09:51.333 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:09:51.333 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:09:51.764 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:09:51.765 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-23T00:09:52.407 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:09:53.334 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:09:53.975 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:09:54.792 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:09:54.793 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-23T00:09:55.169 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:09:55.519 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:09:55.546 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:09:55.574 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:09:55.595 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:09:55.625 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:09:55.649 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:09:55.699 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:09:55.725 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:09:55.751 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:09:55.779 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:09:55.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:09:55.806 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:09:55.807 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:09:55.807 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:09:55.807 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:09:55.807 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:09:55.807 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:09:55.808 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:09:55.808 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:09:55.808 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:09:55.808 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:09:55.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:09:55.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:09:55.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:09:56.540 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:09:56.540 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:09:57.578 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:09:57.913 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:09:58.137 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:09:58.137 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-23T00:09:58.541 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:10:00.432 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:10:00.460 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:10:01.062 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:10:01.063 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-23T00:10:01.089 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:10:01.114 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:10:01.137 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:10:01.162 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:10:01.189 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:10:01.223 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:10:01.254 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:10:01.284 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:10:01.319 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:10:01.351 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:10:01.389 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:10:01.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:10:01.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:10:01.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:10:01.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:10:01.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:10:01.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:10:01.390 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:10:01.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:10:01.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:10:01.391 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:10:01.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:10:01.391 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:10:01.392 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:10:02.258 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:10:02.259 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:10:03.525 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:10:04.042 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:10:04.043 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-23T00:10:04.259 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:10:04.396 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:10:06.714 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:10:07.116 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:10:07.228 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:10:07.254 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:10:07.255 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-23T00:10:08.095 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:10:08.126 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:10:08.156 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:10:08.196 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:10:08.237 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:10:08.272 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:10:08.302 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:10:08.330 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:10:08.359 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:10:08.387 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:10:08.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:10:08.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:10:08.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:10:08.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:10:08.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:10:08.388 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:10:08.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:10:08.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:10:08.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:10:08.389 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:10:08.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:10:08.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:10:08.389 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:10:09.310 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:10:09.310 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:10:09.991 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:10:10.407 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:10:10.508 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:10:10.509 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-23T00:10:11.311 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:10:12.890 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:10:13.002 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:10:13.730 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:10:13.755 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:10:13.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:10:13.802 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:10:13.827 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:10:13.858 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:10:13.883 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:10:13.901 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:10:13.901 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-23T00:10:13.911 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:10:13.937 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:10:13.966 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:10:14.053 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:10:14.054 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:10:14.054 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:10:14.054 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:10:14.054 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:10:14.054 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:10:14.054 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:10:14.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:10:14.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:10:14.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:10:14.055 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:10:14.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:10:14.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:10:14.055 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:10:14.536 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:10:14.536 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:10:16.431 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:10:16.463 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:10:16.544 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:10:17.348 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:10:17.349 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-23T00:10:19.053 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:10:19.101 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:10:19.104 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:10:19.136 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:10:19.165 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:10:19.202 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:10:19.259 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:10:19.309 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:10:19.351 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:10:19.401 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:10:19.432 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:10:19.459 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:10:19.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:10:19.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:10:19.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:10:19.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:10:19.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:10:19.461 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:10:19.461 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:10:19.461 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:10:19.461 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:10:19.461 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:10:19.461 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:10:19.462 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:10:19.462 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:10:19.669 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:10:20.394 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:10:20.395 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:10:20.691 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:10:20.691 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-23T00:10:22.396 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:10:22.454 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:10:23.210 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:10:23.911 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-23T00:10:25.172 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:10:25.314 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:10:25.353 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:10:25.376 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:10:25.401 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:10:25.428 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:10:25.455 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:10:25.484 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:10:25.508 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:10:25.536 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:10:25.563 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:10:25.590 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:10:25.591 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:10:25.591 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:10:25.592 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:10:25.592 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:10:25.592 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:10:25.592 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:10:25.592 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:10:25.592 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:10:25.592 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:10:25.593 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:10:25.593 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:10:25.593 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:10:25.593 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:10:26.110 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:10:26.110 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:10:26.110 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:10:26.111 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:10:26.111 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:10:26.330 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:10:26.330 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:10:26.747 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-23T00:10:28.331 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:10:28.865 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:10:28.865 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:10:28.866 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:10:28.866 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:10:28.866 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:10:29.258 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:10:29.446 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-23T00:10:31.262 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:10:31.286 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:10:31.322 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:10:31.352 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:10:31.384 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:10:31.413 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:10:31.440 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:10:31.467 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:10:31.497 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:10:31.524 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:10:31.551 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:10:31.552 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:10:31.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:10:31.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:10:31.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:10:31.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:10:31.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:10:31.553 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:10:31.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:10:31.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:10:31.554 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:10:31.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:10:31.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:10:31.554 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:10:32.072 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:10:32.115 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:10:32.116 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:10:32.325 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:10:32.503 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-23T00:10:34.117 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:10:34.820 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:10:35.444 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:10:35.450 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-23T00:10:37.190 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:10:37.218 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:10:37.242 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:10:37.276 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:10:37.309 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:10:37.341 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:10:37.370 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:10:37.398 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:10:37.426 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:10:37.456 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:10:37.480 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:10:37.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:10:37.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:10:37.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:10:37.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:10:37.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:10:37.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:10:37.482 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:10:37.483 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:10:37.483 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:10:37.483 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:10:37.483 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:10:37.483 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:10:37.483 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:10:37.959 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:10:37.959 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:10:37.959 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:10:37.960 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:10:37.960 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:10:38.020 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:10:38.020 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:10:38.263 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:10:38.392 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-23T00:10:40.021 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:10:41.040 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:10:41.040 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:10:41.041 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:10:41.041 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:10:41.431 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:10:41.967 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-23T00:10:42.679 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:10:42.719 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:10:42.743 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:10:42.768 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:10:42.794 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:10:42.822 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:10:42.849 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:10:42.879 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:10:42.906 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:10:42.933 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:10:42.960 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:10:42.962 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:10:42.962 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:10:42.962 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:10:42.963 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:10:42.963 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:10:42.963 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:10:42.963 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:10:42.963 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:10:42.963 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:10:42.963 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:10:42.964 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:10:42.964 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:10:42.964 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:10:43.678 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:10:43.678 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:10:44.258 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:10:44.266 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:10:44.933 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-23T00:10:45.679 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:10:47.331 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:10:47.532 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:10:48.397 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-23T00:10:48.798 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:10:48.829 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:10:48.855 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:10:48.881 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:10:48.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:10:48.936 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:10:48.963 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:10:48.990 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:10:49.022 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:10:49.051 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:10:49.081 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:10:49.081 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:10:49.081 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:10:49.081 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:10:49.081 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:10:49.082 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:10:49.082 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:10:49.082 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:10:49.082 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:10:49.082 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:10:49.082 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:10:49.082 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:10:49.083 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:10:49.083 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:10:49.878 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:10:49.878 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:10:50.138 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:10:50.620 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:10:50.620 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:10:50.621 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:10:50.621 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:10:51.317 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-23T00:10:51.880 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:10:53.463 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:10:53.779 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:10:53.779 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:10:53.779 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:10:53.780 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:10:53.780 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:10:54.554 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-23T00:10:54.941 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:10:55.791 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:10:55.844 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:10:55.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:10:55.941 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:10:55.974 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:10:56.015 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:10:56.047 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:10:56.076 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:10:56.105 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:10:56.132 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:10:56.134 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:10:56.134 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:10:56.134 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:10:56.135 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:10:56.135 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:10:56.135 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:10:56.135 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:10:56.135 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:10:56.135 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:10:56.135 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:10:56.135 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:10:56.136 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:10:56.136 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:10:56.312 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:10:56.687 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:10:56.687 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:10:56.857 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:10:57.458 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-23T00:10:58.687 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:10:59.269 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:11:00.169 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:11:01.107 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:11:01.107 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-23T00:11:01.508 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:11:02.030 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:11:02.360 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:11:02.395 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:11:02.433 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:11:02.477 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:11:02.511 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:11:02.563 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:11:02.604 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:11:02.645 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:11:02.678 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:11:02.718 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:11:02.720 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:11:02.720 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:11:02.720 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:11:02.720 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:11:02.721 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:11:02.721 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:11:02.721 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:11:02.721 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:11:02.721 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:11:02.721 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:11:02.721 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:11:02.722 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:11:02.722 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:11:03.346 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:11:03.346 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:11:03.509 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:11:04.161 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:11:04.161 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-23T00:11:05.347 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:11:05.472 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:11:06.424 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:11:06.971 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:11:06.971 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-23T00:11:08.060 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:11:08.347 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:11:08.370 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:11:08.399 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:11:08.424 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:11:08.451 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:11:08.477 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:11:08.504 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:11:08.531 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:11:08.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:11:08.592 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:11:08.624 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:11:08.625 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:11:08.626 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:11:08.626 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:11:08.626 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:11:08.626 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:11:08.626 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:11:08.627 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:11:08.627 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:11:08.627 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:11:08.627 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:11:08.627 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:11:08.628 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:11:08.628 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:11:09.409 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:11:09.410 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:11:09.953 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:11:10.510 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:11:10.510 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-23T00:11:11.411 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:11:11.453 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:11:12.755 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:11:13.527 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:11:13.528 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-23T00:11:13.914 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:11:13.944 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:11:13.983 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:11:14.034 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:11:14.076 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:11:14.107 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:11:14.137 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:11:14.165 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:11:14.199 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:11:14.226 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:11:14.229 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:11:14.254 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:11:14.256 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:11:14.256 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:11:14.257 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:11:14.257 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:11:14.257 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:11:14.257 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:11:14.257 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:11:14.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:11:14.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:11:14.258 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:11:14.258 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:11:14.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:11:14.259 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:11:14.936 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:11:14.937 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:11:16.167 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:11:16.771 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:11:16.772 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-23T00:11:16.938 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:11:17.362 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:11:19.397 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:11:19.615 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:11:19.633 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:11:19.656 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:11:19.680 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:11:19.706 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:11:19.734 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:11:19.761 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:11:19.788 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:11:19.819 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:11:19.841 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:11:19.866 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:11:19.868 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:11:19.868 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:11:19.868 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:11:19.868 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:11:19.868 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:11:19.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:11:19.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:11:19.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:11:19.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:11:19.869 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:11:19.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:11:19.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:11:19.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:11:19.952 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:11:19.953 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-23T00:11:20.314 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:11:20.314 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:11:20.659 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:11:22.315 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:11:22.428 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:11:23.101 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:11:23.102 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-23T00:11:23.457 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:11:25.359 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:11:25.391 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:11:25.421 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:11:25.447 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:11:25.475 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:11:25.503 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:11:25.528 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:11:25.555 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:11:25.581 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:11:25.591 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:11:25.607 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:11:25.634 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:11:25.636 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:11:25.636 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:11:25.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:11:25.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:11:25.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:11:25.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:11:25.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:11:25.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:11:25.638 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:11:25.638 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:11:25.638 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:11:25.638 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:11:25.638 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:11:26.240 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:11:26.240 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-23T00:11:26.318 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:11:26.413 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:11:26.413 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:11:28.414 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:11:28.997 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:11:29.449 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:11:29.581 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:11:29.581 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-23T00:11:31.018 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:11:31.045 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:11:31.089 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:11:31.110 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:11:31.144 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:11:31.193 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:11:31.236 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:11:31.276 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:11:31.322 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:11:31.347 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:11:31.381 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:11:31.383 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:11:31.383 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:11:31.384 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:11:31.384 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:11:31.384 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:11:31.385 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:11:31.385 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:11:31.385 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:11:31.386 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:11:31.386 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:11:31.386 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:11:31.386 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:11:31.387 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:11:31.958 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:11:31.958 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:11:32.364 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:11:32.390 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:11:32.880 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:11:32.881 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-23T00:11:33.959 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:11:35.512 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:11:36.021 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:11:36.652 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:11:36.653 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-23T00:11:36.904 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:11:36.928 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:11:36.951 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:11:36.973 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:11:37.000 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:11:37.027 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:11:37.053 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:11:37.080 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:11:37.107 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:11:37.135 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:11:37.163 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:11:37.164 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:11:37.164 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:11:37.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:11:37.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:11:37.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:11:37.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:11:37.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:11:37.165 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:11:37.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:11:37.166 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:11:37.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:11:37.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:11:37.166 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:11:37.743 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:11:37.744 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:11:38.100 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:11:38.742 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:11:39.303 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:11:39.304 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-23T00:11:39.745 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:11:41.487 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:11:41.879 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:11:42.778 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:11:42.778 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-23T00:11:42.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:11:42.838 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:11:42.864 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:11:42.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:11:42.919 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:11:42.945 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:11:42.970 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:11:42.997 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:11:43.024 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:11:43.051 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:11:43.077 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:11:43.078 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:11:43.079 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:11:43.079 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:11:43.079 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:11:43.079 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:11:43.079 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:11:43.079 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:11:43.080 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:11:43.080 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:11:43.080 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:11:43.080 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:11:43.080 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:11:43.080 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:11:43.752 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:11:43.752 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:11:44.340 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:11:44.706 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:11:45.510 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:11:45.511 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-23T00:11:45.753 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:11:47.444 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:11:48.478 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:11:48.589 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:11:48.612 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:11:48.635 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:11:48.659 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:11:48.685 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:11:48.711 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:11:48.737 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:11:48.763 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:11:48.791 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:11:48.818 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:11:48.844 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:11:48.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:11:48.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:11:48.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:11:48.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:11:48.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:11:48.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:11:48.846 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:11:48.847 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:11:48.847 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:11:48.847 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:11:48.847 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:11:48.847 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:11:48.847 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:11:48.998 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:11:48.999 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-23T00:11:49.402 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:11:49.402 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:11:50.430 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:11:50.978 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:11:51.403 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:11:51.551 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:11:51.551 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-23T00:11:53.507 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:11:54.589 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:11:54.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:11:54.637 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:11:54.661 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:11:54.687 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:11:54.734 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:11:54.781 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:11:54.822 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:11:54.843 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:11:54.848 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:11:54.893 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:11:54.920 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:11:54.922 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:11:54.922 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:11:54.922 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:11:54.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:11:54.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:11:54.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:11:54.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:11:54.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:11:54.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:11:54.923 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:11:54.923 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:11:54.924 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:11:54.924 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:11:55.517 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:11:55.518 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-23T00:11:55.678 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:11:55.678 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:11:56.208 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:11:57.622 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:11:57.679 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:11:58.491 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:11:58.492 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-23T00:11:59.375 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:12:00.881 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:12:01.153 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:12:01.721 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:12:01.721 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-23T00:12:01.732 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:12:01.774 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:12:01.811 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:12:01.845 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:12:01.878 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:12:01.908 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:12:01.935 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:12:01.962 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:12:01.990 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:12:02.016 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:12:02.018 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:12:02.018 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:12:02.018 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:12:02.018 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:12:02.019 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:12:02.019 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:12:02.019 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:12:02.019 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:12:02.019 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:12:02.019 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:12:02.019 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:12:02.020 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:12:02.020 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:12:02.101 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:12:02.890 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:12:02.891 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:12:04.340 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:12:04.892 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:12:04.919 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:12:04.919 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-23T00:12:05.568 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:12:07.518 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:12:07.869 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:12:08.370 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:12:08.411 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-23T00:12:08.413 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:12:08.427 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:12:08.467 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:12:08.499 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:12:08.529 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:12:08.564 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:12:08.588 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:12:08.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:12:08.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:12:08.683 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:12:08.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:12:08.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:12:08.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:12:08.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:12:08.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:12:08.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:12:08.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:12:08.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:12:08.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:12:08.686 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:12:08.686 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:12:08.686 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:12:08.686 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:12:09.543 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:12:09.544 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:12:11.089 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:12:11.545 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:12:11.596 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-23T00:12:11.596 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:12:14.094 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:12:14.125 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:12:14.406 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:12:14.406 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:12:14.407 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:12:14.407 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:12:14.407 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:12:14.989 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:12:15.039 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:12:15.091 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:12:15.096 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-23T00:12:15.127 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:12:15.157 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:12:15.185 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:12:15.212 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:12:15.238 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:12:15.271 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:12:15.296 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:12:15.298 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:12:15.298 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:12:15.298 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:12:15.298 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:12:15.298 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:12:15.299 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:12:15.299 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:12:15.299 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:12:15.299 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:12:15.299 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:12:15.299 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:12:15.299 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:12:15.300 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:12:15.962 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:12:15.962 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:12:17.284 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:12:17.545 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:12:17.546 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:12:17.546 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:12:17.546 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:12:17.963 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:12:18.259 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-23T00:12:19.900 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:12:20.299 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:12:20.326 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:12:20.346 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:12:20.370 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:12:20.409 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:12:20.465 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:12:20.465 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:12:20.475 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:12:20.502 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:12:20.504 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:12:20.530 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:12:20.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:12:20.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:12:20.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:12:20.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:12:20.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:12:20.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:12:20.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:12:20.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:12:20.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:12:20.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:12:20.560 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:12:20.560 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:12:20.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:12:20.561 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:12:21.113 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-23T00:12:21.759 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:12:21.759 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:12:23.500 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:12:23.513 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:12:23.760 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:12:24.147 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-23T00:12:26.331 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:12:26.430 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:12:26.620 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:12:27.283 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:12:27.290 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-23T00:12:27.326 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:12:27.363 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:12:27.397 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:12:27.442 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:12:27.480 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:12:27.514 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:12:27.542 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:12:27.570 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:12:27.597 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:12:27.599 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:12:27.599 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:12:27.599 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:12:27.600 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:12:27.600 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:12:27.600 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:12:27.600 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:12:27.600 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:12:27.600 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:12:27.601 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:12:27.601 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:12:27.601 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:12:27.601 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:12:28.373 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:12:28.373 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:12:29.486 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:12:29.630 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:12:29.630 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:12:29.631 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:12:29.631 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:12:29.631 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:12:30.344 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-23T00:12:30.375 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:12:32.177 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:12:32.676 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:12:32.677 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:12:32.677 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:12:32.677 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:12:32.677 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:12:32.957 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:12:32.994 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:12:33.018 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:12:33.046 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:12:33.069 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:12:33.098 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:12:33.125 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:12:33.152 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:12:33.177 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:12:33.206 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:12:33.231 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:12:33.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:12:33.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:12:33.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:12:33.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:12:33.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:12:33.233 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:12:33.234 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:12:33.234 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:12:33.234 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:12:33.234 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:12:33.234 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:12:33.234 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:12:33.234 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:12:33.471 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-23T00:12:34.070 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:12:34.070 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:12:35.545 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:12:36.072 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:12:36.233 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:12:36.692 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-23T00:12:38.156 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:12:38.829 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:12:38.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:12:38.876 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:12:38.900 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:12:38.908 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:12:38.908 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:12:38.909 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:12:38.909 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:12:38.909 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:12:38.927 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:12:38.953 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:12:38.979 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:12:39.033 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:12:39.056 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:12:39.090 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:12:39.124 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:12:39.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:12:39.125 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:12:39.126 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:12:39.126 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:12:39.126 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:12:39.126 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:12:39.127 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:12:39.127 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:12:39.127 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:12:39.127 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:12:39.128 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:12:39.128 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:12:39.128 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:12:39.580 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-23T00:12:40.123 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:12:40.124 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:12:41.507 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:12:42.105 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:12:42.105 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:12:42.105 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:12:42.106 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:12:42.106 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:12:42.124 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:12:42.669 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-23T00:12:44.141 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:12:44.314 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:12:44.338 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:12:44.367 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:12:44.391 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:12:44.420 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:12:44.448 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:12:44.476 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:12:44.512 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:12:44.547 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:12:44.573 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:12:44.598 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:12:44.600 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:12:44.600 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:12:44.601 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:12:44.601 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:12:44.601 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:12:44.601 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:12:44.601 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:12:44.601 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:12:44.601 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:12:44.602 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:12:44.602 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:12:44.602 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:12:44.602 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:12:45.253 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:12:45.284 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:12:45.285 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:12:45.767 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:12:45.767 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-23T00:12:47.286 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:12:47.483 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:12:48.019 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:12:48.684 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:12:48.685 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-23T00:12:50.275 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:12:50.505 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:12:50.526 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:12:50.612 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:12:50.612 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:12:50.613 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:12:50.626 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:12:50.652 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:12:50.678 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:12:50.705 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:12:50.737 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:12:50.776 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:12:50.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:12:50.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:12:50.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:12:50.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:12:50.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:12:50.777 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:12:50.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:12:50.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:12:50.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:12:50.778 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:12:50.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:12:50.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:12:50.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:12:50.928 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:12:51.456 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:12:51.457 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:12:51.938 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:12:51.939 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-23T00:12:53.458 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:12:53.535 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:12:54.331 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:12:54.970 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:12:54.971 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-23T00:12:56.292 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:12:56.292 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:12:56.293 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:12:56.300 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:12:56.308 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:12:56.337 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:12:56.368 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:12:56.396 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:12:56.423 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:12:56.450 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:12:56.487 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:12:56.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:12:56.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:12:56.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:12:56.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:12:56.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:12:56.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:12:56.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:12:56.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:12:56.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:12:56.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:12:56.519 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:12:56.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:12:56.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:12:56.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:12:57.413 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:12:57.413 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:12:57.656 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:12:58.387 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:12:58.387 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-23T00:12:59.290 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:12:59.414 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:13:00.915 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:13:01.601 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:13:01.602 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-23T00:13:02.098 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:13:02.201 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:13:02.237 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:13:02.268 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:13:02.299 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:13:02.328 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:13:02.358 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:13:02.388 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:13:02.413 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:13:02.457 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:13:02.466 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:13:02.492 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:13:02.494 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:13:02.494 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:13:02.494 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:13:02.494 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:13:02.495 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:13:02.495 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:13:02.495 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:13:02.495 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:13:02.495 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:13:02.496 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:13:02.496 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:13:02.496 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:13:02.496 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:13:03.347 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:13:03.348 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:13:04.345 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:13:04.931 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:13:04.931 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-23T00:13:05.286 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:13:05.348 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:13:07.639 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:13:07.878 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:13:07.902 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:13:07.925 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:13:07.949 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:13:07.975 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:13:08.002 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:13:08.026 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:13:08.032 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:13:08.064 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:13:08.094 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:13:08.121 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:13:08.147 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:13:08.148 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:13:08.149 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:13:08.149 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:13:08.149 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:13:08.149 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:13:08.150 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:13:08.150 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:13:08.150 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:13:08.150 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:13:08.151 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:13:08.151 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:13:08.151 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:13:08.151 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:13:08.282 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:13:08.282 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-23T00:13:08.801 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:13:08.801 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:13:10.802 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:13:10.825 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:13:11.359 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:13:11.360 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-23T00:13:11.546 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:13:13.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:13:13.521 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:13:13.552 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:13:13.581 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:13:13.611 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:13:13.639 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:13:13.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:13:13.696 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:13:13.723 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:13:13.751 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:13:13.776 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:13:13.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:13:13.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:13:13.778 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:13:13.779 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:13:13.779 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:13:13.779 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:13:13.779 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:13:13.779 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:13:13.779 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:13:13.779 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:13:13.780 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:13:13.780 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:13:13.780 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:13:14.197 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:13:14.268 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:13:14.386 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:13:14.387 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:13:14.793 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:13:14.793 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-23T00:13:16.387 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:13:17.260 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:13:17.296 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:13:18.011 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:13:18.011 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-23T00:13:19.343 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:13:20.202 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:13:20.247 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:13:20.259 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:13:20.285 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:13:20.318 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:13:20.359 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:13:20.396 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:13:20.432 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:13:20.466 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:13:20.484 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:13:20.501 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:13:20.528 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:13:20.530 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:13:20.530 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:13:20.531 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:13:20.531 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:13:20.531 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:13:20.531 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:13:20.531 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:13:20.531 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:13:20.531 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:13:20.532 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:13:20.532 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:13:20.532 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:13:20.532 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:13:20.808 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:13:20.809 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-23T00:13:21.294 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:13:21.295 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:13:23.296 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:13:23.352 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:13:23.536 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:13:23.904 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:13:23.904 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-23T00:13:26.261 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:13:26.288 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:13:26.311 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:13:26.321 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:13:26.335 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:13:26.361 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:13:26.388 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:13:26.470 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:13:26.471 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:13:26.472 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:13:26.483 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:13:26.511 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:13:26.539 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:13:26.540 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:13:26.541 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:13:26.541 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:13:26.541 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:13:26.541 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:13:26.542 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:13:26.542 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:13:26.542 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:13:26.542 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:13:26.542 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:13:26.543 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:13:26.543 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:13:26.543 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:13:26.895 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:13:26.895 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-23T00:13:27.382 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:13:27.382 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:13:29.383 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:13:29.426 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:13:29.516 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:13:30.162 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:13:30.163 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-23T00:13:32.332 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:13:32.414 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:13:32.554 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:13:33.185 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:13:33.240 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:13:33.240 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-23T00:13:33.244 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:13:33.277 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:13:33.313 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:13:33.345 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:13:33.378 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:13:33.405 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:13:33.432 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:13:33.459 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:13:33.484 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:13:33.486 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:13:33.486 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:13:33.486 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:13:33.486 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:13:33.486 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:13:33.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:13:33.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:13:33.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:13:33.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:13:33.487 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:13:33.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:13:33.487 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:13:33.488 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:13:34.209 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:13:34.209 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:13:35.598 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:13:36.210 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:13:36.238 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:13:36.764 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:13:36.764 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-23T00:13:38.241 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:13:38.951 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:13:39.033 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:13:39.033 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:13:39.033 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:13:39.034 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:13:39.034 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:13:39.036 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:13:39.064 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:13:39.103 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:13:39.132 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:13:39.159 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:13:39.185 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:13:39.187 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:13:39.187 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:13:39.187 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:13:39.187 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:13:39.187 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:13:39.187 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:13:39.188 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:13:39.188 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:13:39.188 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:13:39.188 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:13:39.188 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:13:39.188 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:13:39.188 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:13:39.609 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:13:39.610 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-23T00:13:40.114 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:13:40.114 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:13:41.538 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:13:42.115 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:13:42.222 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:13:42.779 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:13:42.780 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-23T00:13:44.144 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:13:44.887 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:13:44.914 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:13:44.915 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:13:44.936 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:13:44.961 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:13:44.988 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:13:45.014 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:13:45.040 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:13:45.066 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:13:45.092 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:13:45.127 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:13:45.153 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:13:45.154 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:13:45.155 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:13:45.155 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:13:45.155 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:13:45.155 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:13:45.155 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:13:45.155 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:13:45.155 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:13:45.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:13:45.156 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:13:45.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:13:45.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:13:45.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:13:45.737 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:13:45.737 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-23T00:13:46.036 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:13:46.073 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:13:47.575 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:13:48.074 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:13:48.332 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:13:48.998 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:13:48.998 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-23T00:13:50.124 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:13:50.827 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:13:50.855 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:13:50.881 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:13:50.914 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:13:50.940 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:13:50.967 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:13:50.994 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:13:51.020 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:13:51.047 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:13:51.074 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:13:51.110 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:13:51.158 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:13:51.159 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:13:51.159 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:13:51.159 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:13:51.159 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:13:51.159 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:13:51.159 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:13:51.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:13:51.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:13:51.160 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:13:51.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:13:51.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:13:51.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:13:51.718 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:13:51.719 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:13:51.818 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:13:51.819 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:13:51.819 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:13:51.819 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:13:52.532 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-23T00:13:53.575 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:13:53.722 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:13:55.225 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:13:55.225 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:13:55.226 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:13:55.226 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:13:55.226 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:13:56.020 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-23T00:13:56.409 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:13:56.432 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:13:56.454 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:13:56.478 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:13:56.504 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:13:56.530 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:13:56.556 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:13:56.561 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:13:56.583 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:13:56.610 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:13:56.643 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:13:56.697 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:13:56.699 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:13:56.699 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:13:56.699 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:13:56.699 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:13:56.700 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:13:56.700 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:13:56.700 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:13:56.700 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:13:56.701 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:13:56.701 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:13:56.701 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:13:56.701 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:13:56.701 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:13:57.997 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:13:57.997 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:13:58.772 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:13:59.307 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-23T00:13:59.366 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:13:59.998 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:14:01.874 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:14:01.886 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:14:02.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:14:02.590 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:14:02.591 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-23T00:14:02.614 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:14:02.640 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:14:02.666 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:14:02.694 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:14:02.725 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:14:02.755 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:14:02.787 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:14:02.817 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:14:02.848 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:14:02.866 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:14:02.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:14:02.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:14:02.867 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:14:02.868 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:14:02.868 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:14:02.868 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:14:02.868 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:14:02.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:14:02.869 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:14:02.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:14:02.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:14:02.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:14:03.843 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:14:03.843 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:14:05.160 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:14:05.160 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:14:05.160 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:14:05.160 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:14:05.160 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:14:05.554 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:14:05.650 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-23T00:14:05.845 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:14:08.024 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:14:08.046 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:14:08.046 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:14:08.046 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:14:08.046 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:14:08.046 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:14:08.316 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:14:08.338 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:14:08.362 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:14:08.386 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:14:08.414 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:14:08.449 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:14:08.472 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:14:08.493 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:14:08.546 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:14:08.586 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:14:08.625 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:14:08.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:14:08.637 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:14:08.638 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:14:08.638 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:14:08.638 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:14:08.638 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:14:08.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:14:08.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:14:08.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:14:08.639 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:14:08.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:14:08.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:14:08.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:14:08.783 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-23T00:14:09.383 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:14:09.384 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:14:11.299 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:14:11.385 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:14:11.582 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:14:11.821 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-23T00:14:14.436 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:14:14.492 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:14:14.979 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-23T00:14:17.172 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:14:17.305 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:14:17.648 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-23T00:14:19.687 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:14:19.996 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:14:20.219 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-23T00:14:22.451 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:14:22.569 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:14:23.089 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-23T00:14:24.163 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:14:24.203 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:14:24.252 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:14:24.292 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:14:24.325 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:14:24.354 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:14:24.382 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:14:24.410 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:14:24.441 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:14:24.487 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:14:24.516 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:14:24.517 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:14:24.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:14:24.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:14:24.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:14:24.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:14:24.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:14:24.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:14:24.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:14:24.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:14:24.519 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:14:24.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:14:24.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:14:24.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:14:25.171 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:14:25.171 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:14:25.247 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:14:25.403 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:14:25.404 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:14:25.404 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:14:25.404 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:14:25.404 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:14:25.887 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-23T00:14:27.172 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:14:28.165 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:14:28.318 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:14:29.031 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:14:29.031 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-23T00:14:30.013 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:14:30.035 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:14:30.058 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:14:30.081 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:14:30.107 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:14:30.133 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:14:30.160 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:14:30.186 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:14:30.213 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:14:30.240 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:14:30.266 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:14:30.269 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:14:30.269 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:14:30.269 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:14:30.269 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:14:30.269 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:14:30.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:14:30.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:14:30.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:14:30.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:14:30.270 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:14:30.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:14:30.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:14:30.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:14:30.991 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:14:30.991 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:14:31.275 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:14:31.399 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:14:31.878 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:14:31.878 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-23T00:14:32.992 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:14:34.597 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:14:34.745 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:14:35.604 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:14:35.604 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-23T00:14:35.993 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:14:36.016 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:14:36.040 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:14:36.064 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:14:36.089 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:14:36.115 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:14:36.141 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:14:36.185 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:14:36.224 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:14:36.276 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:14:36.317 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:14:36.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:14:36.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:14:36.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:14:36.319 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:14:36.320 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:14:36.320 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:14:36.320 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:14:36.320 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:14:36.321 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:14:36.321 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:14:36.321 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:14:36.322 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:14:36.322 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:14:37.166 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:14:37.167 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:14:37.334 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:14:37.954 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:14:38.597 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:14:38.598 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-23T00:14:39.168 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:14:40.575 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:14:40.939 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:14:41.932 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:14:41.932 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-23T00:14:43.616 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:14:43.896 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:14:44.662 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:14:44.663 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-23T00:14:46.609 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:14:47.133 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:14:47.611 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:14:47.612 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-23T00:14:49.165 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:14:49.541 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:14:50.306 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:14:50.306 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-23T00:14:52.290 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:14:52.316 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:14:52.336 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:14:52.339 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:14:52.364 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:14:52.390 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:14:52.442 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:14:52.490 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:14:52.522 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:14:52.553 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:14:52.584 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:14:52.596 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:14:52.613 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:14:52.615 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:14:52.615 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:14:52.615 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:14:52.616 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:14:52.616 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:14:52.616 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:14:52.616 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:14:52.616 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:14:52.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:14:52.617 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:14:52.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:14:52.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:14:52.617 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:14:52.899 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:14:52.899 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-23T00:14:53.689 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:14:53.689 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:14:55.249 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:14:55.455 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:14:55.690 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:14:55.769 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:14:55.769 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-23T00:14:58.649 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:14:58.671 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:14:58.699 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:14:58.723 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:14:58.749 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:14:58.777 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:14:58.803 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:14:58.829 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:14:58.856 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:14:58.879 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:14:58.882 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:14:58.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:14:58.911 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:14:58.911 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:14:58.911 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:14:58.911 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:14:58.911 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:14:58.911 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:14:58.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:14:58.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:14:58.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:14:58.912 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:14:58.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:14:58.912 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:14:58.913 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:14:59.318 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:14:59.599 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:14:59.599 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-23T00:14:59.885 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:14:59.885 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:15:01.633 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:15:01.886 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:15:02.075 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:15:02.121 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:15:02.122 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-23T00:15:04.743 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:15:04.781 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:15:04.814 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:15:04.839 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:15:04.865 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:15:04.893 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:15:04.921 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:15:04.948 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:15:04.973 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:15:05.000 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:15:05.026 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:15:05.028 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:15:05.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:15:05.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:15:05.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:15:05.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:15:05.029 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:15:05.030 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:15:05.030 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:15:05.030 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:15:05.030 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:15:05.031 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:15:05.031 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:15:05.031 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:15:05.127 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:15:05.597 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:15:05.666 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:15:05.667 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:15:05.847 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:15:05.848 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-23T00:15:07.668 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:15:07.851 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:15:08.219 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:15:08.496 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:15:08.496 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-23T00:15:10.792 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:15:10.828 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:15:10.869 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:15:10.888 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:15:10.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:15:10.934 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:15:10.957 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:15:10.987 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:15:11.014 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:15:11.039 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:15:11.065 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:15:11.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:15:11.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:15:11.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:15:11.066 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:15:11.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:15:11.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:15:11.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:15:11.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:15:11.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:15:11.067 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:15:11.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:15:11.068 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:15:11.068 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:15:11.549 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:15:11.618 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:15:11.631 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:15:11.632 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:15:12.175 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:15:12.175 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-23T00:15:13.633 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:15:14.062 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:15:14.345 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:15:14.679 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:15:14.680 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-23T00:15:16.576 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:15:16.604 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:15:16.634 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:15:16.658 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:15:16.685 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:15:16.710 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:15:16.753 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:15:16.797 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:15:16.833 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:15:16.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:15:16.899 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:15:16.901 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:15:16.901 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:15:16.902 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:15:16.902 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:15:16.902 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:15:16.902 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:15:16.903 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:15:16.903 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:15:16.903 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:15:16.903 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:15:16.903 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:15:16.904 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:15:16.904 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:15:17.323 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:15:17.486 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:15:17.486 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:15:18.163 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:15:18.645 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:15:18.645 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-23T00:15:19.487 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:15:19.917 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:15:20.813 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:15:21.728 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:15:21.728 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-23T00:15:22.374 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:15:22.395 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:15:22.423 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:15:22.443 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:15:22.471 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:15:22.501 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:15:22.528 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:15:22.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:15:22.583 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:15:22.611 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:15:22.638 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:15:22.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:15:22.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:15:22.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:15:22.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:15:22.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:15:22.639 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:15:22.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:15:22.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:15:22.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:15:22.640 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:15:22.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:15:22.640 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:15:22.641 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:15:23.560 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:15:23.560 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:15:23.626 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:15:24.251 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:15:24.682 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:15:24.683 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-23T00:15:25.561 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:15:26.330 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:15:26.730 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:15:27.609 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:15:27.609 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-23T00:15:28.284 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:15:28.334 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:15:28.375 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:15:28.411 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:15:28.446 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:15:28.474 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:15:28.504 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:15:28.547 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:15:28.580 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:15:28.610 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:15:28.662 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:15:28.664 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:15:28.664 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:15:28.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:15:28.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:15:28.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:15:28.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:15:28.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:15:28.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:15:28.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:15:28.666 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:15:28.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:15:28.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:15:28.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:15:29.430 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:15:29.430 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:15:30.212 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:15:30.607 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:15:30.748 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:15:30.757 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-23T00:15:31.431 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:15:33.025 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:15:33.125 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:15:33.797 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-23T00:15:33.974 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:15:33.998 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:15:34.022 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:15:34.047 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:15:34.073 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:15:34.098 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:15:34.125 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:15:34.154 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:15:34.182 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:15:34.208 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:15:34.235 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:15:34.237 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:15:34.237 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:15:34.237 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:15:34.237 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:15:34.237 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:15:34.237 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:15:34.237 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:15:34.238 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:15:34.238 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:15:34.238 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:15:34.238 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:15:34.238 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:15:34.238 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:15:35.102 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:15:35.103 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:15:36.198 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:15:36.198 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:15:36.198 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:15:36.198 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:15:36.199 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:15:36.688 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-23T00:15:36.854 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:15:37.104 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:15:39.310 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:15:39.310 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:15:39.310 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:15:39.311 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:15:39.311 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:15:39.418 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:15:39.554 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:15:39.577 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:15:39.601 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:15:39.624 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:15:39.650 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:15:39.676 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:15:39.703 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:15:39.730 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:15:39.756 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:15:39.782 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:15:39.809 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:15:39.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:15:39.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:15:39.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:15:39.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:15:39.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:15:39.812 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:15:39.812 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:15:39.812 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:15:39.812 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:15:39.812 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:15:39.812 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:15:39.812 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:15:39.813 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:15:39.941 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-23T00:15:40.471 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:15:40.471 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:15:42.388 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:15:42.472 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:15:42.874 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:15:42.921 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-23T00:15:45.297 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:15:45.510 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:15:45.547 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:15:45.547 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:15:45.548 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:15:45.548 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:15:45.548 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:15:46.136 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-23T00:15:46.159 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:15:46.204 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:15:46.241 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:15:46.275 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:15:46.307 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:15:46.336 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:15:46.363 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:15:46.391 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:15:46.420 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:15:46.457 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:15:46.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:15:46.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:15:46.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:15:46.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:15:46.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:15:46.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:15:46.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:15:46.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:15:46.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:15:46.459 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:15:46.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:15:46.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:15:46.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:15:47.646 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:15:47.647 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:15:48.612 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:15:48.612 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:15:48.612 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:15:48.613 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:15:48.613 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:15:49.146 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-23T00:15:49.321 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:15:49.647 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:15:51.692 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:15:51.693 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:15:51.693 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:15:51.693 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:15:51.873 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:15:52.193 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-23T00:15:52.217 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:15:52.249 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:15:52.275 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:15:52.298 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:15:52.325 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:15:52.351 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:15:52.378 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:15:52.404 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:15:52.435 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:15:52.462 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:15:52.496 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:15:52.498 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:15:52.498 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:15:52.498 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:15:52.498 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:15:52.498 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:15:52.498 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:15:52.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:15:52.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:15:52.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:15:52.499 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:15:52.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:15:52.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:15:52.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:15:53.282 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:15:53.283 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:15:54.929 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:15:55.283 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:15:55.459 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-23T00:15:55.636 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:15:57.652 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:15:58.309 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:15:58.529 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:15:58.537 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:15:58.583 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:15:58.626 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:15:58.663 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:15:58.700 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:15:58.731 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:15:58.760 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:15:58.788 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:15:58.815 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:15:58.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:15:58.847 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:15:58.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:15:58.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:15:58.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:15:58.848 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:15:58.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:15:58.849 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:15:58.851 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:15:58.851 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:15:58.852 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:15:58.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:15:58.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:15:58.852 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:15:58.977 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-23T00:15:59.395 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:15:59.396 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:16:01.397 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:16:01.398 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:16:01.398 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:16:01.398 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:16:01.398 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:16:01.629 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:16:01.961 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-23T00:16:04.382 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:16:04.527 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:16:04.606 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:16:04.607 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:16:04.607 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:16:04.607 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:16:04.607 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:16:05.127 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-23T00:16:05.234 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:16:05.273 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:16:05.309 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:16:05.335 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:16:05.382 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:16:05.409 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:16:05.436 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:16:05.463 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:16:05.489 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:16:05.516 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:16:05.517 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:16:05.517 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:16:05.517 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:16:05.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:16:05.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:16:05.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:16:05.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:16:05.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:16:05.518 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:16:05.518 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:16:05.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:16:05.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:16:05.519 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:16:06.364 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:16:06.365 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:16:07.650 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:16:08.181 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-23T00:16:08.340 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:16:08.366 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:16:10.793 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:16:11.129 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:16:11.279 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:16:11.313 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:16:11.313 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:16:11.314 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-23T00:16:11.336 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:16:11.360 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:16:11.389 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:16:11.436 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:16:11.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:16:11.517 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:16:11.549 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:16:11.580 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:16:11.612 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:16:11.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:16:11.614 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:16:11.614 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:16:11.614 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:16:11.614 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:16:11.614 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:16:11.615 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:16:11.615 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:16:11.615 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:16:11.615 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:16:11.615 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:16:11.616 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:16:11.616 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:16:12.329 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:16:12.329 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:16:13.994 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:16:14.330 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:16:14.495 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:16:14.496 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-23T00:16:14.625 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:16:17.072 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:16:17.106 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:16:17.130 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:16:17.162 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:16:17.194 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:16:17.223 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:16:17.250 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:16:17.280 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:16:17.285 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:16:17.303 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:16:17.331 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:16:17.357 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:16:17.358 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:16:17.358 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:16:17.358 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:16:17.358 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:16:17.359 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:16:17.359 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:16:17.359 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:16:17.359 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:16:17.359 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:16:17.359 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:16:17.359 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:16:17.360 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:16:17.360 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:16:17.439 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:16:17.991 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:16:17.991 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-23T00:16:18.289 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:16:18.290 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:16:20.291 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:16:20.372 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:16:20.641 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:16:21.036 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:16:21.036 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-23T00:16:23.168 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:16:23.192 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:16:23.214 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:16:23.237 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:16:23.264 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:16:23.290 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:16:23.317 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:16:23.331 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:16:23.342 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:16:23.369 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:16:23.396 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:16:23.422 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:16:23.423 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:16:23.423 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:16:23.423 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:16:23.424 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:16:23.424 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:16:23.424 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:16:23.424 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:16:23.424 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:16:23.424 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:16:23.424 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:16:23.425 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:16:23.425 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:16:23.425 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:16:23.474 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:16:24.103 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:16:24.104 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-23T00:16:24.270 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:16:24.271 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:16:26.271 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:16:26.516 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:16:26.639 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:16:27.194 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:16:27.194 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-23T00:16:28.998 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:16:29.030 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:16:29.057 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:16:29.091 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:16:29.124 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:16:29.154 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:16:29.185 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:16:29.216 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:16:29.243 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:16:29.271 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:16:29.303 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:16:29.305 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:16:29.305 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:16:29.305 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:16:29.306 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:16:29.306 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:16:29.306 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:16:29.306 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:16:29.307 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:16:29.307 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:16:29.307 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:16:29.307 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:16:29.308 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:16:29.308 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:16:29.566 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:16:29.643 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:16:29.917 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:16:29.917 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:16:30.302 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:16:30.302 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-23T00:16:31.918 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:16:32.565 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:16:32.667 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:16:33.313 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:16:33.314 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-23T00:16:34.643 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:16:34.679 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:16:34.711 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:16:34.745 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:16:34.771 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:16:34.803 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:16:34.838 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:16:34.872 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:16:34.904 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:16:34.934 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:16:34.977 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:16:34.978 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:16:34.978 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:16:34.978 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:16:34.979 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:16:34.979 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:16:34.980 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:16:34.980 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:16:34.980 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:16:34.980 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:16:34.980 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:16:34.981 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:16:34.981 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:16:34.981 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:16:35.588 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:16:35.588 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:16:35.593 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:16:35.829 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:16:36.498 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:16:36.498 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-23T00:16:37.589 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:16:38.702 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:16:38.992 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:16:39.875 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:16:39.875 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-23T00:16:40.434 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:16:40.458 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:16:40.481 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:16:40.504 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:16:40.545 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:16:40.570 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:16:40.596 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:16:40.623 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:16:40.649 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:16:40.676 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:16:40.703 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:16:40.704 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:16:40.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:16:40.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:16:40.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:16:40.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:16:40.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:16:40.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:16:40.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:16:40.706 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:16:40.706 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:16:40.706 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:16:40.706 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:16:40.706 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:16:41.452 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:16:41.497 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:16:41.498 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:16:42.060 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:16:42.629 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:16:42.630 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-23T00:16:43.498 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:16:44.712 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:16:45.171 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:16:45.896 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:16:45.896 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-23T00:16:46.287 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:16:46.309 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:16:46.331 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:16:46.355 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:16:46.381 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:16:46.407 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:16:46.433 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:16:46.461 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:16:46.487 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:16:46.514 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:16:46.557 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:16:46.557 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:16:46.557 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:16:46.557 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:16:46.558 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:16:46.558 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:16:46.558 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:16:46.558 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:16:46.558 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:16:46.558 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:16:46.558 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:16:46.559 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:16:46.569 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:16:46.570 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:16:47.260 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:16:47.260 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:16:47.636 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:16:48.021 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:16:48.701 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:16:48.701 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-23T00:16:49.261 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:16:50.660 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:16:51.616 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:16:51.718 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:16:51.740 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:16:51.763 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:16:51.786 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:16:51.812 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:16:51.840 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:16:51.864 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:16:51.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:16:51.922 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:16:51.944 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:16:51.970 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:16:51.972 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:16:51.973 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:16:51.973 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:16:51.973 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:16:51.973 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:16:51.973 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:16:51.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:16:51.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:16:51.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:16:51.974 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:16:51.974 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:16:51.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:16:51.975 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:16:52.100 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:16:52.100 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-23T00:16:52.451 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:16:52.451 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:16:53.555 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:16:53.923 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:16:54.453 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:16:54.522 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:16:54.522 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-23T00:16:56.371 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:16:57.641 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:16:57.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:16:57.920 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:16:57.938 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:16:57.961 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:16:57.987 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:16:58.014 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:16:58.039 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:16:58.065 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:16:58.091 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:16:58.118 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:16:58.145 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:16:58.147 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:16:58.159 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:16:58.159 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:16:58.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:16:58.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:16:58.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:16:58.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:16:58.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:16:58.160 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:16:58.161 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:16:58.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:16:58.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:16:58.161 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:16:58.317 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:16:58.318 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-23T00:16:58.703 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:16:58.704 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:16:59.195 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:17:00.657 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:17:00.705 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:17:01.417 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:17:01.417 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-23T00:17:02.679 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:17:03.839 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:17:03.862 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:17:03.885 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:17:03.908 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:17:03.935 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:17:03.966 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:17:03.996 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:17:04.024 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:17:04.042 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:17:04.049 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:17:04.076 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:17:04.102 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:17:04.104 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:17:04.104 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:17:04.104 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:17:04.104 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:17:04.104 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:17:04.105 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:17:04.105 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:17:04.105 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:17:04.105 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:17:04.105 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:17:04.105 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:17:04.105 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:17:04.106 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:17:04.699 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:17:04.699 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-23T00:17:04.850 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:17:04.851 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:17:05.419 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:17:06.853 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:17:06.925 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:17:07.588 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:17:07.588 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-23T00:17:08.696 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:17:09.728 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:17:09.754 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:17:09.778 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:17:09.802 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:17:09.829 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:17:09.856 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:17:09.885 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:17:09.909 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:17:09.935 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:17:09.975 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:17:10.017 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:17:10.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:17:10.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:17:10.023 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:17:10.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:17:10.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:17:10.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:17:10.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:17:10.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:17:10.024 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:17:10.025 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:17:10.036 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:17:10.037 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:17:10.037 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:17:10.518 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:17:10.608 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:17:10.609 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:17:10.989 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:17:10.989 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-23T00:17:11.660 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:17:12.610 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:17:13.130 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:17:14.009 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:17:14.010 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-23T00:17:15.644 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:17:15.729 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:17:16.500 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:17:16.529 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:17:16.554 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:17:16.581 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:17:16.612 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:17:16.617 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:17:16.644 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:17:16.676 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:17:16.706 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:17:16.734 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:17:16.760 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:17:16.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:17:16.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:17:16.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:17:16.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:17:16.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:17:16.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:17:16.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:17:16.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:17:16.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:17:16.766 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:17:16.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:17:16.767 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:17:16.767 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:17:17.126 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-23T00:17:17.504 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:17:17.505 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:17:18.368 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:17:19.108 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:17:19.507 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:17:19.657 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-23T00:17:21.678 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:17:22.184 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:17:22.206 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:17:22.230 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:17:22.258 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:17:22.293 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:17:22.324 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:17:22.353 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:17:22.380 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:17:22.407 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:17:22.437 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:17:22.463 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:17:22.498 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:17:22.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:17:22.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:17:22.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:17:22.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:17:22.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:17:22.499 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:17:22.500 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:17:22.500 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:17:22.500 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:17:22.500 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:17:22.500 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:17:22.500 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:17:22.953 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:17:22.954 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:17:22.992 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:17:22.992 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:17:22.993 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:17:22.993 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:17:22.993 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:17:23.485 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-23T00:17:24.260 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:17:24.955 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:17:25.702 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:17:25.702 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:17:25.702 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:17:25.702 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:17:26.505 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-23T00:17:27.679 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:17:28.145 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:17:28.999 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:17:29.024 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:17:29.048 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:17:29.093 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:17:29.130 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:17:29.169 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:17:29.200 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:17:29.232 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:17:29.262 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:17:29.292 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:17:29.320 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:17:29.322 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:17:29.322 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:17:29.323 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:17:29.323 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:17:29.323 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:17:29.323 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:17:29.324 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:17:29.324 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:17:29.324 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:17:29.325 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:17:29.325 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:17:29.325 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:17:29.326 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:17:29.545 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-23T00:17:29.901 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:17:29.902 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:17:30.480 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:17:31.866 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:17:31.903 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:17:32.628 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-23T00:17:33.725 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:17:35.095 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:17:35.118 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:17:35.143 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:17:35.173 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:17:35.193 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:17:35.197 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:17:35.197 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:17:35.197 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:17:35.197 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:17:35.219 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:17:35.245 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:17:35.278 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:17:35.298 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:17:35.344 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:17:35.378 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:17:35.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:17:35.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:17:35.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:17:35.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:17:35.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:17:35.381 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:17:35.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:17:35.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:17:35.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:17:35.382 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:17:35.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:17:35.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:17:35.382 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:17:35.697 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-23T00:17:36.048 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:17:36.048 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:17:36.540 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:17:38.017 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:17:38.017 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:17:38.017 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:17:38.017 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:17:38.017 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:17:38.049 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:17:38.812 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-23T00:17:39.791 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:17:41.034 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:17:41.058 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:17:41.081 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:17:41.106 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:17:41.132 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:17:41.173 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:17:41.188 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:17:41.212 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:17:41.228 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:17:41.238 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:17:41.265 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:17:41.293 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:17:41.293 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:17:41.294 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:17:41.294 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:17:41.294 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:17:41.294 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:17:41.294 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:17:41.295 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:17:41.295 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:17:41.295 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:17:41.295 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:17:41.295 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:17:41.296 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:17:41.296 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:17:41.899 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-23T00:17:42.061 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:17:42.061 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:17:42.564 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:17:44.061 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:17:44.180 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:17:44.963 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-23T00:17:45.689 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:17:47.384 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:17:47.421 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:17:47.454 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:17:47.485 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:17:47.516 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:17:47.544 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:17:47.571 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:17:47.597 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:17:47.624 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:17:47.631 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:17:47.631 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:17:47.631 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:17:47.632 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:17:47.632 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:17:47.650 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:17:47.677 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:17:47.678 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:17:47.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:17:47.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:17:47.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:17:47.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:17:47.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:17:47.679 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:17:47.680 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:17:47.680 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:17:47.680 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:17:47.680 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:17:47.680 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:17:47.680 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:17:48.305 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-23T00:17:48.472 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:17:48.473 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:17:48.500 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:17:50.474 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:17:50.633 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:17:50.634 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:17:50.634 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:17:50.634 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:17:50.635 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:17:51.316 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-23T00:17:51.700 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:17:53.514 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:17:53.550 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:17:53.581 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:17:53.607 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:17:53.634 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:17:53.660 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:17:53.686 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:17:53.712 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:17:53.735 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:17:53.764 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:17:53.801 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:17:53.831 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:17:53.833 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:17:53.833 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:17:53.834 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:17:53.834 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:17:53.834 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:17:53.835 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:17:53.835 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:17:53.835 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:17:53.835 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:17:53.836 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:17:53.836 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:17:53.840 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:17:53.841 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:17:54.360 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:17:54.361 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-23T00:17:54.521 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:17:54.521 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:17:54.780 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:17:56.523 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:17:56.706 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:17:57.356 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:17:57.465 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:17:57.465 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-23T00:17:59.318 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:17:59.343 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:17:59.381 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:17:59.417 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:17:59.453 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:17:59.484 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:17:59.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:17:59.564 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:17:59.606 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:17:59.644 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:17:59.681 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:17:59.687 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:17:59.687 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:17:59.687 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:17:59.687 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:17:59.687 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:17:59.688 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:17:59.688 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:17:59.688 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:17:59.688 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:17:59.688 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:17:59.688 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:17:59.689 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:17:59.689 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:17:59.920 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:18:00.152 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:18:00.228 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:18:00.228 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:18:00.807 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:18:00.808 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-23T00:18:02.229 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:18:02.719 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:18:03.813 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:18:04.429 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:18:04.430 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-23T00:18:05.468 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:18:06.408 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:18:06.991 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:18:06.992 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-23T00:18:09.075 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:18:09.141 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:18:09.731 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:18:09.731 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-23T00:18:11.664 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:18:11.906 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:18:12.203 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:18:12.203 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-23T00:18:14.559 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:18:14.699 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:18:14.855 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:18:14.895 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:18:14.930 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:18:14.999 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:18:15.019 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:18:15.057 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:18:15.103 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:18:15.146 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:18:15.188 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:18:15.206 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:18:15.206 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-23T00:18:15.225 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:18:15.257 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:18:15.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:18:15.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:18:15.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:18:15.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:18:15.270 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:18:15.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:18:15.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:18:15.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:18:15.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:18:15.271 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:18:15.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:18:15.271 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:18:15.272 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:18:15.768 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:18:15.769 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:18:17.296 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:18:17.380 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:18:17.770 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:18:17.843 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:18:17.844 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-23T00:18:20.719 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:18:20.864 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:18:20.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:18:20.920 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:18:20.942 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:18:20.964 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:18:20.991 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:18:21.016 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:18:21.041 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:18:21.047 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:18:21.071 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:18:21.094 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:18:21.120 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:18:21.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:18:21.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:18:21.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:18:21.122 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:18:21.123 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:18:21.123 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:18:21.123 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:18:21.123 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:18:21.123 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:18:21.123 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:18:21.123 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:18:21.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:18:21.124 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:18:21.825 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:18:21.825 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-23T00:18:21.982 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:18:21.983 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:18:23.320 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:18:23.681 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:18:23.984 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:18:24.300 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:18:24.300 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-23T00:18:26.783 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:18:26.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:18:26.827 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:18:26.852 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:18:26.878 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:18:26.914 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:18:26.946 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:18:26.977 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:18:27.022 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:18:27.060 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:18:27.093 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:18:27.094 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:18:27.094 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:18:27.095 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:18:27.095 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:18:27.095 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:18:27.095 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:18:27.095 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:18:27.095 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:18:27.095 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:18:27.096 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:18:27.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:18:27.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:18:27.096 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:18:27.196 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:18:27.723 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:18:27.750 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:18:27.751 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:18:27.950 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:18:27.951 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-23T00:18:29.752 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:18:29.897 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:18:30.314 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:18:30.562 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:18:30.563 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-23T00:18:32.881 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:18:32.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:18:32.941 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:18:32.965 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:18:33.003 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:18:33.017 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:18:33.048 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:18:33.074 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:18:33.100 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:18:33.138 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:18:33.153 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:18:33.155 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:18:33.155 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:18:33.155 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:18:33.155 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:18:33.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:18:33.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:18:33.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:18:33.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:18:33.156 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:18:33.156 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:18:33.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:18:33.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:18:33.157 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:18:33.714 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:18:33.737 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:18:33.759 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:18:33.759 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:18:34.312 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:18:34.313 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-23T00:18:35.760 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:18:36.291 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:18:36.492 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:18:36.875 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:18:36.876 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-23T00:18:38.572 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:18:39.419 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:18:39.456 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:18:39.490 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:18:39.521 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:18:39.550 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:18:39.577 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:18:39.604 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:18:39.629 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:18:39.656 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:18:39.663 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:18:39.682 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:18:39.683 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:18:39.683 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:18:39.683 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:18:39.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:18:39.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:18:39.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:18:39.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:18:39.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:18:39.684 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:18:39.684 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:18:39.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:18:39.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:18:39.685 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:18:39.820 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:18:40.539 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:18:40.539 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-23T00:18:40.540 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:18:40.540 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:18:42.470 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:18:42.543 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:18:42.723 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:18:42.991 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:18:42.991 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-23T00:18:45.528 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:18:46.029 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:18:46.384 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:18:46.431 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:18:46.472 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:18:46.527 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:18:46.561 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:18:46.561 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-23T00:18:46.581 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:18:46.616 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:18:46.645 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:18:46.674 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:18:46.709 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:18:46.732 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:18:46.744 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:18:46.745 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:18:46.745 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:18:46.745 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:18:46.745 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:18:46.745 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:18:46.745 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:18:46.746 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:18:46.746 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:18:46.746 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:18:46.746 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:18:46.746 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:18:46.746 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:18:46.854 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:18:47.666 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:18:47.666 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:18:48.818 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:18:49.289 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:18:49.289 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-23T00:18:49.667 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:18:49.672 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:18:52.275 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:18:52.733 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:18:52.770 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:18:52.837 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:18:52.837 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-23T00:18:53.652 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:18:53.696 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:18:53.729 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:18:53.760 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:18:53.791 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:18:53.818 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:18:53.844 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:18:53.871 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:18:53.898 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:18:53.924 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:18:53.925 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:18:53.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:18:53.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:18:53.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:18:53.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:18:53.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:18:53.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:18:53.926 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:18:53.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:18:53.927 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:18:53.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:18:53.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:18:53.927 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:18:54.549 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:18:54.549 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:18:54.964 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:18:55.174 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:18:55.444 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:18:55.445 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-23T00:18:56.550 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:18:57.773 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:18:57.952 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:18:58.495 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-23T00:18:59.833 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:19:00.695 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:19:00.743 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:19:00.749 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:19:00.750 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:19:00.750 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:19:00.750 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:19:00.750 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:19:00.782 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:19:00.838 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:19:00.868 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:19:00.899 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:19:00.937 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:19:00.954 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:19:00.981 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:19:00.989 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:19:01.008 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:19:01.011 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:19:01.011 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:19:01.011 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:19:01.011 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:19:01.011 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:19:01.012 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:19:01.012 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:19:01.012 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:19:01.012 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:19:01.012 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:19:01.012 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:19:01.012 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:19:01.013 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:19:01.238 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-23T00:19:01.693 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:19:01.694 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:19:03.694 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:19:03.739 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:19:03.757 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:19:03.758 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:19:03.758 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:19:03.758 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:19:03.758 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:19:04.645 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-23T00:19:06.396 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:19:06.593 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:19:06.621 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:19:06.657 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:19:06.684 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:19:06.708 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:19:06.740 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:19:06.766 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:19:06.787 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:19:06.813 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:19:06.813 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:19:06.841 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:19:06.867 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:19:06.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:19:06.869 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:19:06.870 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:19:06.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:19:06.871 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:19:06.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:19:06.872 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:19:06.873 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:19:06.873 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:19:06.873 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:19:06.874 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:19:06.874 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:19:06.875 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:19:07.533 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-23T00:19:07.775 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:19:07.775 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:19:09.372 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:19:09.776 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:19:09.976 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:19:10.576 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-23T00:19:11.966 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:19:12.007 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:19:12.029 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:19:12.052 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:19:12.076 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:19:12.102 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:19:12.129 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:19:12.155 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:19:12.182 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:19:12.209 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:19:12.235 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:19:12.262 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:19:12.263 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:19:12.263 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:19:12.263 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:19:12.263 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:19:12.264 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:19:12.264 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:19:12.264 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:19:12.264 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:19:12.264 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:19:12.264 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:19:12.265 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:19:12.265 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:19:12.265 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:19:13.092 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:19:13.092 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:19:13.669 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:19:13.669 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:19:13.669 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:19:13.669 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:19:13.669 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:19:14.298 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-23T00:19:15.093 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:19:15.769 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:19:16.903 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:19:16.903 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:19:16.904 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:19:16.904 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:19:17.564 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-23T00:19:17.818 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:19:17.840 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:19:17.863 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:19:17.887 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:19:17.912 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:19:17.937 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:19:17.965 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:19:17.991 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:19:18.021 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:19:18.067 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:19:18.099 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:19:18.101 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:19:18.101 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:19:18.101 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:19:18.101 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:19:18.101 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:19:18.101 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:19:18.102 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:19:18.102 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:19:18.102 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:19:18.102 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:19:18.102 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:19:18.102 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:19:18.102 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:19:18.538 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:19:18.683 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:19:18.683 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:19:19.839 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:19:20.604 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-23T00:19:20.685 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:19:21.755 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:19:23.244 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:19:23.450 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:19:23.469 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:19:23.492 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:19:23.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:19:23.542 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:19:23.573 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:19:23.597 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:19:23.623 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:19:23.650 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:19:23.677 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:19:23.703 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:19:23.704 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:19:23.704 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:19:23.704 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:19:23.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:19:23.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:19:23.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:19:23.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:19:23.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:19:23.705 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:19:23.706 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:19:23.706 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:19:23.706 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:19:23.706 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:19:23.781 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-23T00:19:24.187 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:19:24.187 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:19:24.779 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:19:26.075 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:19:26.075 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:19:26.075 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:19:26.075 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:19:26.189 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:19:26.717 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-23T00:19:27.848 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:19:29.357 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:19:29.395 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:19:29.428 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:19:29.459 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:19:29.495 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:19:29.525 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:19:29.554 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:19:29.583 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:19:29.584 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:19:29.609 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:19:29.637 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:19:29.664 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:19:29.664 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:19:29.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:19:29.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:19:29.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:19:29.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:19:29.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:19:29.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:19:29.665 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:19:29.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:19:29.666 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:19:29.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:19:29.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:19:29.666 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:19:30.287 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-23T00:19:30.446 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:19:30.446 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:19:30.622 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:19:32.448 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:19:32.745 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:19:33.426 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-23T00:19:33.920 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:19:35.275 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:19:35.309 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:19:35.341 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:19:35.371 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:19:35.401 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:19:35.428 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:19:35.458 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:19:35.503 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:19:35.544 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:19:35.581 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:19:35.608 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:19:35.611 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:19:35.611 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:19:35.611 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:19:35.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:19:35.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:19:35.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:19:35.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:19:35.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:19:35.612 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:19:35.612 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:19:35.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:19:35.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:19:35.613 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:19:36.263 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:19:36.263 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:19:36.529 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:19:36.813 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:19:37.111 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:19:37.111 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-23T00:19:38.265 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:19:39.561 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:19:39.954 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:19:40.414 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:19:40.415 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-23T00:19:41.070 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:19:41.101 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:19:41.118 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:19:41.141 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:19:41.168 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:19:41.195 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:19:41.222 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:19:41.248 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:19:41.274 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:19:41.301 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:19:41.327 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:19:41.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:19:41.328 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:19:41.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:19:41.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:19:41.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:19:41.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:19:41.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:19:41.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:19:41.329 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:19:41.330 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:19:41.330 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:19:41.330 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:19:41.330 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:19:42.096 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:19:42.096 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:19:42.624 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:19:42.700 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:19:43.144 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:19:43.145 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-23T00:19:44.097 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:19:45.773 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:19:46.253 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:19:46.870 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:19:46.871 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-23T00:19:47.201 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:19:47.222 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:19:47.245 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:19:47.269 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:19:47.295 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:19:47.329 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:19:47.349 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:19:47.387 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:19:47.426 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:19:47.467 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:19:47.500 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:19:47.501 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:19:47.501 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:19:47.501 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:19:47.501 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:19:47.501 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:19:47.501 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:19:47.501 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:19:47.502 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:19:47.502 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:19:47.502 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:19:47.502 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:19:47.502 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:19:47.502 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:19:48.243 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:19:48.243 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:19:48.325 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:19:48.986 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:19:49.502 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:19:49.502 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-23T00:19:50.244 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:19:51.767 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:19:52.040 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:19:52.846 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:19:52.846 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-23T00:19:53.220 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:19:53.243 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:19:53.267 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:19:53.291 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:19:53.317 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:19:53.344 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:19:53.370 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:19:53.397 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:19:53.424 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:19:53.451 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:19:53.479 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:19:53.479 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:19:53.479 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:19:53.480 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:19:53.480 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:19:53.480 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:19:53.480 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:19:53.480 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:19:53.480 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:19:53.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:19:53.481 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:19:53.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:19:53.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:19:53.481 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:19:54.221 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:19:54.221 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:19:54.354 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:19:54.924 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:19:55.524 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:19:55.524 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-23T00:19:56.222 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:19:57.388 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:19:57.737 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:19:58.687 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:19:58.688 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-23T00:19:59.229 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:20:00.085 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:20:00.124 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:20:00.156 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:20:00.186 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:20:00.221 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:20:00.238 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:20:00.289 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:20:00.330 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:20:00.366 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:20:00.395 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:20:00.421 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:20:00.422 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:20:00.422 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:20:00.422 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:20:00.422 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:20:00.422 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:20:00.422 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:20:00.423 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:20:00.423 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:20:00.423 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:20:00.423 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:20:00.423 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:20:00.423 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:20:00.423 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:20:00.546 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:20:01.397 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:20:01.398 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:20:01.568 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:20:01.568 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-23T00:20:03.399 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:20:03.781 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:20:03.871 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:20:04.585 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:20:04.586 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-23T00:20:06.140 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:20:06.170 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:20:06.195 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:20:06.226 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:20:06.257 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:20:06.284 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:20:06.312 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:20:06.340 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:20:06.384 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:20:06.421 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:20:06.457 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:20:06.458 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:20:06.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:20:06.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:20:06.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:20:06.459 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:20:06.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:20:06.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:20:06.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:20:06.460 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:20:06.461 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:20:06.461 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:20:06.461 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:20:06.461 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:20:06.950 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:20:06.955 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:20:07.029 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:20:07.045 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:20:07.550 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:20:07.551 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-23T00:20:09.045 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:20:09.744 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:20:09.766 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:20:10.909 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:20:10.909 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-23T00:20:11.704 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:20:11.734 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:20:11.761 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:20:11.786 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:20:11.814 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:20:11.846 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:20:11.876 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:20:11.905 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:20:11.932 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:20:11.957 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:20:11.983 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:20:11.985 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:20:11.985 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:20:11.985 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:20:11.985 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:20:11.985 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:20:11.985 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:20:11.986 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:20:11.986 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:20:11.986 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:20:11.986 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:20:11.986 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:20:11.986 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:20:11.987 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:20:12.439 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:20:12.777 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:20:12.778 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:20:13.201 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:20:13.708 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:20:13.708 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-23T00:20:14.779 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:20:15.798 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:20:15.919 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:20:16.916 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:20:16.917 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-23T00:20:17.484 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:20:17.509 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:20:17.531 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:20:17.554 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:20:17.607 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:20:17.652 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:20:17.682 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:20:17.712 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:20:17.743 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:20:17.769 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:20:17.805 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:20:17.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:20:17.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:20:17.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:20:17.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:20:17.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:20:17.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:20:17.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:20:17.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:20:17.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:20:17.811 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:20:17.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:20:17.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:20:17.811 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:20:18.501 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:20:18.501 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:20:18.918 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:20:19.007 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:20:19.522 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:20:19.522 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-23T00:20:20.505 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:20:21.784 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:20:22.229 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:20:23.042 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:20:23.042 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_blocked_ops 2019-12-23T00:20:23.446 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:20:24.301 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:20:24.361 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:20:24.407 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:20:24.439 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:20:24.476 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:20:24.497 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:20:24.525 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:20:24.551 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:20:24.583 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:20:24.605 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:20:24.607 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:20:24.607 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:20:24.607 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:20:24.607 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:20:24.607 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:20:24.608 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:20:24.608 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:20:24.608 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:20:24.608 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:20:24.608 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:20:24.608 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:20:24.609 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:20:24.609 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:20:24.766 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:20:25.055 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:20:25.384 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:20:25.384 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:20:25.722 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:20:25.722 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_historic_ops 2019-12-23T00:20:27.385 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:20:27.785 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:20:28.035 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:20:28.966 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:20:28.967 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_ops_in_flight 2019-12-23T00:20:29.775 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:20:30.632 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:20:30.665 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:20:30.667 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:20:30.700 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:20:30.742 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:20:30.779 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:20:30.822 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:20:30.858 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:20:30.884 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:20:30.914 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:20:30.939 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:20:30.944 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:20:30.945 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:20:30.945 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:20:30.945 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:20:30.945 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:20:30.945 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:20:30.946 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:20:30.947 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:20:30.947 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:20:30.947 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:20:30.947 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:20:30.948 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:20:30.948 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:20:31.049 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:20:31.985 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:20:31.986 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:20:31.986 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:20:31.987 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_blocked_ops 2019-12-23T00:20:33.830 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:20:33.987 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:20:34.587 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:20:35.158 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:20:35.159 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.10 dump_historic_ops 2019-12-23T00:20:36.379 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:20:36.555 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:20:37.232 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:20:37.422 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:20:37.465 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:20:37.500 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:20:37.533 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:20:37.565 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:20:37.595 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:20:37.638 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:20:37.681 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:20:37.714 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:20:37.746 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:20:37.748 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:20:37.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:20:37.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:20:37.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:20:37.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:20:37.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:20:37.749 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:20:37.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:20:37.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:20:37.750 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:20:37.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:20:37.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:20:37.750 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:20:37.884 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:20:37.885 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_ops_in_flight 2019-12-23T00:20:38.714 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:20:38.714 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:20:40.150 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:20:40.420 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:20:40.444 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:20:40.444 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:20:40.444 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:20:40.715 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:20:41.030 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_blocked_ops 2019-12-23T00:20:42.692 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:20:42.973 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:20:43.002 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:20:43.029 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:20:43.061 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:20:43.091 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:20:43.118 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:20:43.144 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:20:43.172 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:20:43.199 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:20:43.223 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:20:43.250 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:20:43.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:20:43.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:20:43.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:20:43.253 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:20:43.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:20:43.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:20:43.254 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:20:43.256 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:20:43.256 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:20:43.257 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:20:43.257 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:20:43.257 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:20:43.257 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:20:43.276 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:20:44.042 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:20:44.042 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:20:44.350 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.1 dump_historic_ops 2019-12-23T00:20:45.811 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:20:46.044 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:20:46.685 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:20:47.250 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_ops_in_flight 2019-12-23T00:20:48.412 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:20:48.415 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:20:48.438 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:20:48.461 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:20:48.486 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:20:48.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:20:48.541 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:20:48.574 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:20:48.593 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:20:48.619 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:20:48.647 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:20:48.673 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:20:48.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:20:48.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:20:48.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:20:48.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:20:48.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:20:48.675 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:20:48.676 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:20:48.676 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:20:48.676 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:20:48.676 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:20:48.676 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:20:48.676 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:20:48.676 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:20:49.351 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:20:49.351 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:20:50.147 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:20:50.148 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:20:50.148 INFO:teuthology.orchestra.run.smithi091.stdout: "num_ops": 0 2019-12-23T00:20:50.148 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:20:50.714 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_blocked_ops 2019-12-23T00:20:51.353 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:20:51.804 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:20:53.596 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:20:53.596 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:20:53.597 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:20:53.597 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:20:53.597 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:20:53.835 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:20:53.853 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:20:53.877 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:20:53.901 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:20:53.926 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:20:53.957 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:20:53.979 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:20:54.007 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:20:54.032 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:20:54.059 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:20:54.085 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:20:54.089 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:20:54.089 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:20:54.090 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:20:54.090 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:20:54.090 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:20:54.090 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:20:54.091 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:20:54.091 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:20:54.091 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:20:54.091 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:20:54.092 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:20:54.092 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:20:54.092 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:20:54.226 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.0 dump_historic_ops 2019-12-23T00:20:54.624 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:20:54.625 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:20:54.792 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:20:56.628 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:20:56.724 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:20:57.414 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_ops_in_flight 2019-12-23T00:20:57.848 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:20:59.605 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:20:59.858 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:21:00.439 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_blocked_ops 2019-12-23T00:21:00.474 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:21:00.526 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:21:00.559 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:21:00.591 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:21:00.621 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:21:00.650 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:21:00.682 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:21:00.710 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:21:00.737 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:21:00.763 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:21:00.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:21:00.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:21:00.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:21:00.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:21:00.765 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:21:00.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:21:00.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:21:00.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:21:00.766 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:21:00.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:21:00.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:21:00.766 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:21:00.858 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:21:01.454 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:21:01.455 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:21:02.790 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:21:02.791 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:21:02.791 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:21:02.791 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:21:02.791 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:21:03.388 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.3 dump_historic_ops 2019-12-23T00:21:03.456 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:21:04.888 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:21:05.835 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:21:05.862 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:21:05.887 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:21:05.911 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:21:05.938 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:21:05.977 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:21:06.012 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:21:06.047 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:21:06.079 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:21:06.109 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:21:06.137 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:21:06.139 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:21:06.139 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:21:06.139 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:21:06.139 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:21:06.139 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:21:06.140 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:21:06.140 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:21:06.140 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:21:06.140 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:21:06.140 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:21:06.140 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:21:06.140 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:21:06.141 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:21:06.678 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:21:06.678 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:21:06.680 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:21:07.126 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_ops_in_flight 2019-12-23T00:21:07.871 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:21:08.679 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:21:09.426 INFO:teuthology.orchestra.run.smithi091.stdout:ERROR: (22) Invalid argument 2019-12-23T00:21:10.245 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_blocked_ops 2019-12-23T00:21:11.397 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:21:11.919 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:21:12.772 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:21:12.818 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:21:12.844 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:21:12.844 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [], 2019-12-23T00:21:12.845 INFO:teuthology.orchestra.run.smithi091.stdout: "complaint_time": 30, 2019-12-23T00:21:12.845 INFO:teuthology.orchestra.run.smithi091.stdout: "num_blocked_ops": 0 2019-12-23T00:21:12.845 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:21:12.851 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:21:12.883 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:21:12.913 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:21:12.942 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:21:12.969 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:21:12.997 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:21:13.027 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:21:13.056 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:21:13.061 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:21:13.061 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:21:13.062 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:21:13.062 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:21:13.062 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:21:13.062 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:21:13.063 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:21:13.063 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:21:13.063 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:21:13.063 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:21:13.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:21:13.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:21:13.064 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:21:13.380 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.2 dump_historic_ops 2019-12-23T00:21:13.792 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:21:13.792 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:21:14.390 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:21:15.664 INFO:teuthology.orchestra.run.smithi091.stdout:{ 2019-12-23T00:21:15.664 INFO:teuthology.orchestra.run.smithi091.stdout: "size": 20, 2019-12-23T00:21:15.664 INFO:teuthology.orchestra.run.smithi091.stdout: "duration": 600, 2019-12-23T00:21:15.664 INFO:teuthology.orchestra.run.smithi091.stdout: "ops": [] 2019-12-23T00:21:15.664 INFO:teuthology.orchestra.run.smithi091.stdout:} 2019-12-23T00:21:15.793 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:21:16.385 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_ops_in_flight 2019-12-23T00:21:17.400 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:21:18.953 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:21:18.979 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:21:19.005 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:21:19.030 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:21:19.065 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:21:19.098 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:21:19.128 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:21:19.157 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:21:19.183 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:21:19.211 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:21:19.212 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:21:19.247 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:21:19.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:21:19.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:21:19.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:21:19.249 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:21:19.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:21:19.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:21:19.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:21:19.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:21:19.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:21:19.250 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:21:19.250 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:21:19.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:21:19.251 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:21:19.867 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:21:19.868 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_blocked_ops 2019-12-23T00:21:20.014 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:21:20.014 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:21:20.256 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:21:22.015 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:21:22.263 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:21:23.050 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:21:23.050 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.5 dump_historic_ops 2019-12-23T00:21:23.835 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:21:24.740 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:21:24.762 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:21:24.794 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:21:24.823 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:21:24.853 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:21:24.882 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:21:24.913 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:21:24.942 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:21:24.990 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:21:25.029 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:21:25.065 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:21:25.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:21:25.067 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:21:25.068 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:21:25.068 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:21:25.068 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:21:25.068 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:21:25.069 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:21:25.069 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:21:25.069 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:21:25.070 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:21:25.070 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:21:25.070 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:21:25.070 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:21:25.708 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:21:25.758 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:21:25.759 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:21:26.171 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:21:26.172 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_ops_in_flight 2019-12-23T00:21:26.797 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:21:27.760 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:21:28.283 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:21:29.306 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:21:29.307 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_blocked_ops 2019-12-23T00:21:30.783 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:21:30.816 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:21:30.834 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:21:30.851 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:21:30.890 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:21:30.925 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:21:30.959 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:21:30.991 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:21:31.021 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:21:31.057 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:21:31.078 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:21:31.105 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:21:31.106 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:21:31.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:21:31.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:21:31.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:21:31.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:21:31.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:21:31.107 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:21:31.108 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:21:31.108 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:21:31.108 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:21:31.108 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:21:31.108 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:21:31.108 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:21:31.783 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:21:31.851 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:21:31.851 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:21:32.248 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:21:32.249 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.4 dump_historic_ops 2019-12-23T00:21:33.404 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:21:33.852 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:21:34.240 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:21:35.184 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:21:35.184 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_ops_in_flight 2019-12-23T00:21:36.864 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:21:36.943 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:21:37.005 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:21:37.041 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:21:37.075 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:21:37.107 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:21:37.137 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:21:37.165 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:21:37.193 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:21:37.220 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:21:37.246 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:21:37.272 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:21:37.274 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:21:37.274 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:21:37.275 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:21:37.275 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:21:37.275 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:21:37.275 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:21:37.275 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:21:37.275 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:21:37.276 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:21:37.276 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:21:37.276 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:21:37.276 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:21:37.276 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:21:37.894 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:21:37.895 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:21:38.231 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:21:38.770 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:21:38.771 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_blocked_ops 2019-12-23T00:21:39.398 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:21:39.896 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:21:41.708 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:21:42.456 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:21:42.457 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.7 dump_historic_ops 2019-12-23T00:21:42.788 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:21:42.811 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:21:42.843 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:21:42.874 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:21:42.882 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is set 2019-12-23T00:21:42.915 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:21:42.943 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:21:42.975 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:21:43.000 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:21:43.028 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:21:43.054 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:21:43.080 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:21:43.081 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'false' 2019-12-23T00:21:43.081 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'false' 2019-12-23T00:21:43.082 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'false' 2019-12-23T00:21:43.082 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'false' 2019-12-23T00:21:43.082 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'false' 2019-12-23T00:21:43.082 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'false' 2019-12-23T00:21:43.082 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'false' 2019-12-23T00:21:43.082 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'false' 2019-12-23T00:21:43.083 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'false' 2019-12-23T00:21:43.083 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:21:43.083 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:21:43.083 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'false' 2019-12-23T00:21:43.083 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'false' 2019-12-23T00:21:44.045 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:21:44.045 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:21:44.790 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:21:45.346 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:21:45.347 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_ops_in_flight 2019-12-23T00:21:45.595 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set nodeep-scrub 2019-12-23T00:21:46.045 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2019-12-23T00:21:48.004 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:21:48.828 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:21:48.828 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_blocked_ops 2019-12-23T00:21:48.971 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is set 2019-12-23T00:21:51.225 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:21:51.552 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset noscrub 2019-12-23T00:21:51.751 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:21:51.751 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.6 dump_historic_ops 2019-12-23T00:21:54.209 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:21:54.725 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:21:54.726 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_ops_in_flight 2019-12-23T00:21:54.993 INFO:teuthology.orchestra.run.smithi091.stderr:noscrub is unset 2019-12-23T00:21:56.971 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:21:57.463 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:21:57.463 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_blocked_ops 2019-12-23T00:21:57.654 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd unset nodeep-scrub 2019-12-23T00:21:59.515 INFO:teuthology.orchestra.run.smithi091.stdout:osd.0: {} 2019-12-23T00:21:59.552 INFO:teuthology.orchestra.run.smithi091.stdout:osd.1: {} 2019-12-23T00:21:59.583 INFO:teuthology.orchestra.run.smithi091.stdout:osd.2: {} 2019-12-23T00:21:59.612 INFO:teuthology.orchestra.run.smithi091.stdout:osd.3: {} 2019-12-23T00:21:59.641 INFO:teuthology.orchestra.run.smithi091.stdout:osd.4: {} 2019-12-23T00:21:59.667 INFO:teuthology.orchestra.run.smithi091.stdout:osd.5: {} 2019-12-23T00:21:59.695 INFO:teuthology.orchestra.run.smithi091.stdout:osd.6: {} 2019-12-23T00:21:59.715 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:21:59.721 INFO:teuthology.orchestra.run.smithi091.stdout:osd.7: {} 2019-12-23T00:21:59.748 INFO:teuthology.orchestra.run.smithi091.stdout:osd.8: {} 2019-12-23T00:21:59.777 INFO:teuthology.orchestra.run.smithi091.stdout:osd.10: {} 2019-12-23T00:21:59.806 INFO:teuthology.orchestra.run.smithi091.stdout:osd.11: {} 2019-12-23T00:21:59.808 INFO:teuthology.orchestra.run.smithi091.stderr:osd.0: osd_enable_op_tracker = 'true' 2019-12-23T00:21:59.808 INFO:teuthology.orchestra.run.smithi091.stderr:osd.1: osd_enable_op_tracker = 'true' 2019-12-23T00:21:59.808 INFO:teuthology.orchestra.run.smithi091.stderr:osd.2: osd_enable_op_tracker = 'true' 2019-12-23T00:21:59.808 INFO:teuthology.orchestra.run.smithi091.stderr:osd.3: osd_enable_op_tracker = 'true' 2019-12-23T00:21:59.808 INFO:teuthology.orchestra.run.smithi091.stderr:osd.4: osd_enable_op_tracker = 'true' 2019-12-23T00:21:59.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.5: osd_enable_op_tracker = 'true' 2019-12-23T00:21:59.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.6: osd_enable_op_tracker = 'true' 2019-12-23T00:21:59.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.7: osd_enable_op_tracker = 'true' 2019-12-23T00:21:59.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.8: osd_enable_op_tracker = 'true' 2019-12-23T00:21:59.809 INFO:teuthology.orchestra.run.smithi091.stderr:Error ENXIO: problem getting command descriptions from osd.9 2019-12-23T00:21:59.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.9: problem getting command descriptions from osd.9 2019-12-23T00:21:59.809 INFO:teuthology.orchestra.run.smithi091.stderr:osd.10: osd_enable_op_tracker = 'true' 2019-12-23T00:21:59.810 INFO:teuthology.orchestra.run.smithi091.stderr:osd.11: osd_enable_op_tracker = 'true' 2019-12-23T00:22:00.350 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:22:00.350 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.8 dump_historic_ops 2019-12-23T00:22:00.518 DEBUG:teuthology.orchestra.run:got remote process result: 6 2019-12-23T00:22:00.518 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2019-12-23T00:22:00.863 INFO:teuthology.orchestra.run.smithi091.stderr:nodeep-scrub is unset 2019-12-23T00:22:02.456 INFO:teuthology.orchestra.run.smithi091.stderr:admin_socket: exception getting command descriptions: [Errno 2] No such file or directory 2019-12-23T00:22:02.520 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2019-12-23T00:22:03.114 DEBUG:teuthology.orchestra.run:got remote process result: 22 2019-12-23T00:22:03.115 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph daemon osd.11 dump_ops_in_flight 2019-12-23T00:22:03.484 INFO:teuthology.orchestra.run.smithi091:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph-ci/ceph:99b509ffd4dd354919b08b38ba83b998fbfdc107 shell -c /home/ubuntu/cephtest/ceph.conf -k /home/ubuntu/cephtest/ceph.keyring --fsid 147630da-2509-11ea-8284-001a4aab830c -- ceph osd set noscrub 2019-12-23T00:22:05.316 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2019-12-23T00:22:05.317 DEBUG:teuthology.task.console_log:Killing console logger for smithi043 2019-12-23T00:22:05.317 DEBUG:teuthology.task.console_log:Killing console logger for smithi091 2019-12-23T00:22:05.318 DEBUG:teuthology.task.console_log:Killing console logger for smithi107 2019-12-23T00:22:05.318 DEBUG:teuthology.task.console_log:Killing console logger for smithi043 2019-12-23T00:22:05.318 DEBUG:teuthology.task.console_log:Killing console logger for smithi091 2019-12-23T00:22:05.319 DEBUG:teuthology.task.console_log:Killing console logger for smithi107 2019-12-23T00:22:05.319 DEBUG:teuthology.exit:Finished running handlers